Senior QA Engineer ID66279

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.

WHY JOIN US

If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!

About The Role

We are looking for a Senior QA Engineer to perform manual testing of web applications and validate AI-generated outputs as part of a product team building LLM-integrated features. You will design and execute test cases across UI, API, and AI behavior scenarios using TestRail, Postman, and JIRA, working within an Agile environment. The role requires a detail-oriented engineer who can evaluate model quality metrics and ensure accuracy, completeness, and alignment across prompt–response workflows.

What You Will Do

• Perform manual testing of web applications to identify bugs, glitches, and inconsistencies;
• Develop, maintain, and execute detailed manual test cases based on product requirements and specifications;
• Perform thorough cross-browser UI testing to ensure consistent behavior across multiple browsers;
• Validate AI-generated test cases, summaries, and recommendations produced by integrated LLMs;
• Design and execute test scenarios to verify AI behavior, including prompt-response consistency and contextual relevance;
• Evaluate AI model quality using defined acceptance metrics such as precision, recall, hallucination rate, and duplication rate;
• Conduct manual API testing to validate request and response accuracy, error handling, and integration;
• Use TestRail to create, manage, and track test plans, test cases, and test execution results;
• Use Jenkins CI/CD pipelines;
• Report and document defects clearly using JIRA;
• Participate in daily stand-ups, sprint planning, and other Agile team meetings;
• Review user stories and acceptance criteria to understand the scope of testing;
• Estimate own tasks and report risks to the team lead;
• Collaborate with developers and other QA team members to reproduce issues and verify fixes;
• Maintain and update test documentation as needed.
  
  

MUST HAVES

• 4+ years of experience with manual testing;
• Basic understanding of methodologies and SDLC and STLC models;
• Knowledge of testing and quality assurance processes;
• Strong experience with test design activities to cover a set of product requirements;
• Knowledge of test design techniques and testing methodologies;
• Defect analysis skills and the ability to write detailed and useful defect reports;
• Experience with Postman for API testing and browser developer tools;
• Experience with UI testing;
• Experience with TestRail as a test management system;
• Basic understanding of CI/CD processes;
• Intermediate English level.
  
  

NICE TO HAVES

• Understanding of LLM evaluation principles, including prompt engineering, output validation, and model fine-tuning cycles;
• Familiarity with AI and ML concepts, APIs, and frameworks such as OpenAI API, Azure OpenAI, or LangChain;
• Knowledge of data protection and compliance standards for AI systems, including GDPR, zero-data retention, and PII anonymization;
• Experience with SSO testing;
• Experience in performance or load testing.
  
  

Perks And Benefits

• Professional growth: Mentorship, TechTalks, and personalized growth roadmaps.
• Competitive compensation: USD-based pay with education, fitness, and team activity budgets.
• Exciting projects: Modern solutions with Fortune 500 and top product companies.
• Flextime: Flexible schedule with remote and office options.