Artificial Intelligence Validation Engineer

Introduction

• As an AI Validation Engineer within a cross-functional team of Engineers, Data Scientists, and Product Owners, you will primarily focus on testing AI-driven chat applications and mobile apps, ensuring their reliability, performance, and seamless functionality. If you are passionate about seeing your ideas evolve from initial concept to shaping engaging AI-powered experiences for millions of users, Avrioc is the perfect place for you!

Requirements

• Proven experience as an AI Validation Engineer with at least 2+ years dedicated to testing AI/ML systems.
• Bachelor's degree in computer science, Engineering, or a related technical field.
• Basic hands on with Python coding.
• Proven expertise in AI/ML testing methodologies, including validation of models like NLP (Specially LLM) , CV, and predictive systems.
• Hands-on experience in automation testing frameworks such as PyTest, Selenium, Robot Framework, or equivalent.
• Strong background in prompt testing – ability to craft both functional and adversarial prompts to stress-test generative AI models.
• Solid understanding of VAPT (Vulnerability Assessment & Penetration Testing) techniques in AI/ML contexts.
• Experience in Testing of LLM`s, jailbreak of LLM`s required.
• Experience with data validation, bias/fairness audits, model drift detection, and performance benchmarking, PyTest , robot framework.
• Having Python experience SQL, or Shell scripting is a plus. (Minimal knowledge is fine)
• Willingness to participate in an on-call rotation, including handling high-priority, time-sensitive issues that may arise during off-hours.
• Ability to function under pressure and support real-time production environments.
• Clear communication skills to report issues, document findings, and coordinate with developers and AI researchers.
• Must be able to generate reports based on testing results.

Nice to have

• Good appreciation for the Agile/Scrum methodology.
• Developer background would be nice to have.

Responsibilities & Authorities

• Design and execute comprehensive test plans for AI-driven applications, including edge cases, performance stress, and failure modes.
• Perform rigorous prompt testing on LLMs and generative AI models to identify weaknesses and ensure behavior aligns with design intent.
• Conduct automated and manual testing to validate model inputs, outputs, and user interactions across platforms.
• Test security, data privacy, and compliance readiness of AI systems through VAPT and audit mechanisms.
• Continuously monitor live AI product performance, identify anomalies, and report bugs in a structured format.
• Support the AI engineering and data science teams by validating models, assisting in debugging, and ensuring stability in staging and production.
• Provide on-call support for critical issues, particularly during non-standard hours, with compensatory time off as per company policy.
• Collaborate in model explain ability, fairness, and accountability testing, ensuring all ethical and regulatory requirements are met.

If you meet the qualifications and are interested in this position, apply for it or please share your resume with us at [email protected] with the subject line: "Your Name | Years of Experience | Notice Period, Location" along with details like your current salary, expected salary, etc.

Please note that only shortlisted candidates will be contacted.

Elevate your career and embark on a transformative journey with us.