Senior Test Automation Engineer – AI Solutions (Python)

We are looking for a talented and experienced Senior Test Automation Engineer to join our team and help productionize AI solutions leveraging cutting-edge technologies. This is an exciting opportunity to work on AI-driven systems, including Retrieval Augmented Generation (RAG), Large Language Models (LLMs), and AWS Bedrock capabilities. You will play a critical role in ensuring the functionality, performance, scalability, security, and reliability of these innovative solutions.

Responsibilities

• Design and implement comprehensive testing strategies tailored to AI workflows, APIs, and cloud-native solutions
• Develop detailed test plans, test cases, and scripts for AI systems, including data pipelines, knowledge bases, vector databases, and LLM-powered components
• Validate the performance and accuracy of LLM outputs, ensuring compliance with prompt engineering best practices and client-specific requirements
• Test RAG-based architectures, including retrieval processes, embedding validation, and semantic search accuracy
• Ensure seamless integration of data ingestion, cleansing, transformation, and validation pipelines with vector databases and AI models
• Build and maintain test automation frameworks for AI components and data pipelines using industry-standard tools and custom scripts
• Conduct performance, load, and scalability testing for AI/LLM endpoints, RAG workflows, and deployed pipelines
• Test cloud infrastructure deployment (e.g., IaC via Terraform/CloudFormation) and ensure production-readiness for AWS Bedrock, S3, EKS, and vector database configurations
• Identify, log, and manage defects in AI workflows, APIs, data pipelines, and backend systems, collaborating with developers and DevOps teams to resolve issues
• Monitor system reliability, availability, and fault tolerance in cloud environments to ensure production-level readiness

Requirements

• Proficiency in test automation using Python (3+ years of experience)
• Proven experience in QA engineering, with a focus on testing AI systems and cloud-native solutions
• Strong knowledge of testing frameworks, tools, and methodologies for AI workflows, APIs, and data pipelines
• Hands-on experience with LLMs, RAG architectures, and vector databases
• Proficiency in test automation using Python or similar scripting languages
• Expertise in performance and scalability testing for AI systems
• Familiarity with cloud environments, particularly AWS services like Bedrock, S3, and EKS
• Solid understanding of Infrastructure as Code (IaC) tools such as Terraform or CloudFormation
• Excellent problem-solving skills and ability to collaborate effectively with cross-functional teams
• Strong verbal and written communication skills in English, at least at a B2 level

Nice to have

• Experience with AWS Bedrock capabilities and generative AI solutions
• Knowledge of semantic search and embedding validation techniques
• Familiarity with continuous integration and delivery pipelines for AI systems
• Background in data engineering or data science

We offer

• Engineering Heritage: Best-in-class experts sharing a culture of engineering excellence and tackling complex engineering challenges for over 30 years.
• Advanced Tech Stack: Innovative projects where you can apply or enhance your expertise in Cloud, Data, AI, and other emerging technologies
• World-Class Clients: Work closely with 295+ of the Forbes Global 2000 on creating disruptive solutions that make a global impact
• Professional Growth: Exceptional support for career development with comprehensive resources for upskilling or reskilling in pioneering practices
• GenAI Community: Strong AI competencies with 600+ experts across 55+ locations driving GenAI-enabled transformation journeys
• Entrepreneurial Culture: If you're passionate and dedicated to improving business transformation, we provide the support you need to bring your ideas to life
• Hybrid Setup: The flexibility to work from any location in Lithuania, whether it's your home or our dynamic offices in Vilnius and Kaunas
• Other Benefits: Additional vacation and trust days, private health insurance, Employee Stock Purchase Plan and more

Salary range €3K-€4.8K gross, based on your experience and interview results.

Join our team in our cozy offices in Vilnius or Kaunas.

About EPAM

EPAM is a leading global provider of digital platform engineering and development services. For over 30 years, our team has helped leading brands navigate the waves of digital transformation, building solutions that help them stay competitive through constant market disruption.

With offices in 55+ countries, EPAM has grown in Lithuania to over 1,200+ talented innovators in just 4 years. We foster creativity and unconventional ways of doing things, welcoming like-minded professionals to join us

Feel free to work remotely from anywhere across Lithuania or connect with colleagues at our Vilnius and Kaunas offices.