Location: Dubai

We’re seeking a highly skilled technical specialist to rigorously assess and validate third-party AI solutions. You will collaborate closely with Product Managers, AI Architects, and domain experts to scrutinize vendor offerings, create innovative evaluation methods, and perform hands-on testing. Your goal is to verify vendor claims, measure performance, and determine the suitability of AI solutions across multiple contexts - Provider, Payer, Enterprise, and Consumer.

Drawing on deep knowledge of AI/ML principles, data analysis, and validation frameworks, you will help the organization invest in AI technologies that are robust, effective, and ethically sound. You’ll play a pivotal role in separating hype from reality and providing the technical evidence needed for strategic decisions.

Key Responsibilities

• Perform comprehensive technical reviews of third-party AI products, including their methodologies, algorithms, data requirements, disclosed model architectures, and performance claims.
• Design and implement rigorous evaluation frameworks and testing protocols for diverse AI applications (predictive analytics, NLP, computer vision, etc.).
• Execute evaluation plans: prepare data, engineer features for comparative analysis when necessary, run vendor models/APIs, and analyze outputs.
• Benchmark and compare competing vendor solutions addressing the same problem.
• Translate complex evaluation results into clear, actionable insights for both technical and business stakeholders.
• Produce detailed technical reports and presentations summarizing findings, highlighting performance, limitations, risks, and data-fit considerations.
• Collaborate with AI Architects and engineering teams to assess integration feasibility and data-pipeline needs.
• Share best practices and contribute to the broader technical community on AI model evaluation and validation.

Minimum Requirements

• Master’s degree or PhD in Data Science, Computer Science, Statistics, Mathematics, Physics, or another quantitative discipline.
• 3–5+ years of hands-on experience as a Data Scientist or Machine Learning Engineer, with a portfolio showcasing AI/ML model development, evaluation, and deployment.
• Strong programming skills in Python or R, along with libraries such as scikit-learn, TensorFlow, PyTorch, Pandas, and NumPy.
• Solid understanding of a broad range of machine learning algorithms - classification, regression, clustering, NLP, and computer vision - and their mathematical foundations.
• Proven track record in model validation techniques, including cross-validation, A/B testing, and bias assessment.
• Advanced statistical analysis abilities and a critical eye for model performance metrics.
• Proficiency with SQL or similar data-querying languages and experience working with large datasets.

Preferred Qualifications

• Background in healthcare (Provider, Payer, Life Sciences, HealthTech) and familiarity with healthcare data such as EHR, claims, imaging, or omics.
• Experience evaluating third-party AI/ML products or working with MLaaS platforms.
• Knowledge of ethical AI practices, including fairness, accountability, and transparency (FAT).
• Familiarity with cloud-based ML platforms such as AWS SageMaker, Azure ML, or Google AI Platform.
• Understanding of data governance, privacy regulations (HIPAA, GDPR), and security in AI contexts.
• Creative problem-solving skills and the ability to overcome evaluation challenges with limited information.
• Excellent communication skills for presenting complex technical findings to varied audiences.