About Artificial Analysis

Artificial Analysis is an independent benchmarking, evaluation and insights provider for AI. Our benchmarks let engineers and companies make the best decisions on which technologies and providers to use, empowering them to build the next generation of AI applications. 

Our public benchmarks are trusted by tens of thousands of users every week and have been cited by publications from TechCrunch to SemiAnalysis to the All-In Podcast (E165: 52:00 to 1:12:00; E167: 28:20 to 31:30). We work with leading companies across the AI industry. 

The job has the potential for remote work with WeWork access provided.

The Opportunity

We're looking for an Intermediate to Senior ML Engineer to join our team and lead projects in AI benchmarking and analysis. You’ll work closely with our founders to build core parts of the software stack for our early stage start-up - if you join in the next few weeks, you’ll likely be joining a team of 10.

The coming wave of AI scaling is going to change the world in ways we don’t yet understand - and we’re offering a front row seat. 

What You'll Do

• Lead the development and optimization of aspects of our core benchmarking stack, focusing on data intensive backend systems and APIs, and driving projects from concept to shipped
• Design and implement robust Python solutions for benchmarking, model evaluation and data analysis
• Design and implement user interfaces and data visualizations that distill complex AI benchmarking data into intuitive, interactive experiences for thousands of daily users
• Conduct custom analyses for enterprise customers, providing actionable insights to inform their AI strategies
• Contribute to the evolution of our benchmarking methodologies, including development of evaluation methodology for emerging modalities and capabilities
• Embrace an AI-native workflow, using cutting-edge AI tools to generate leverage in a fast-changing industry

What We're Looking For

• 3+ years of professional software engineering experience
• Passion for AI and eagerness to work at the forefront of technological innovation
• Proficiency with relevant Python libraries for data analysis (e.g. pandas) and key AI APIs (eg. OpenAI)
• Familiarity with cloud infrastructure, orchestration and monitoring tools.
• Experience with visualizing & presenting data
• Strong problem-solving skills and ability to distill complex concepts into actionable insights in the face of uncertainty
• Excellent communication and collaboration skills
• Proven ability to lead projects independently in a fast-paced environment
• Bachelor's or Master's in Computer Science, Engineering, or related field (eg. Physics)
• Preferred but not essential:
• Experience with AI/ML frameworks (eg. PyTorch)
• Creation of analytical reports

Why Artificial Analysis?

• Work with frontier AI technologies and collaborate with companies building the future of AI
• Make a significant impact in a fast-growing startup
• Competitive compensation package (including potential for equity)
• We offer flexible work arrangements, including remote and hybrid options. Our team is currently split between San Francisco and Sydney.

If you're excited about the challenge of building the future of AI benchmarking and analysis, we'd love to chat. 

To apply for the role, please email [email protected] with (1) your CV (and LinkedIn URL), (2) a brief note on why you're excited about joining Artificial Analysis, and (3) a two sentence description of your favorite AI paper from the last 6 months.