About Artificial Analysis

Artificial Analysis is an independent AI benchmarking and insights company. We support engineers and companies to understand AI capabilities and make critical decisions about their AI strategies.

Our public benchmarks are trusted by hundreds of thousands of users monthly and have been cited by publications from TechCrunch to the Wall Street Journal to the All-In Podcast (E165: 52:00 to 1:12:00; E167: 28:20 to 31:30). Our work is regularly cited by leading AI companies, with recent examples including NVIDIA, Meta, Amazon and Bytedance.

We are backed by Nat Friedman (Github, Meta), Daniel Gross (SSI), Andrew Ng (Google Brain, DeepLearning.ai, Amazon), Adam D’Angelo (Quora, Poe, OpenAI), Clem Delangue (Hugging Face) and other industry leaders.

The Opportunity

We’re hiring for a Product Manager focused on AI media generation (image, video, music, speech and more). You'll work closely with us to develop and enhance our media generation arenas and leaderboards, contributing to product strategy and execution at the intersection of creativity and AI.

This is a hands-on, technical product role: you’ll drive day-to-day operations for our media generation offering and be a key contributor to product development, collaborating closely with engineering and analysis colleagues, as well as our founders. You’ll write and run experiments, curate datasets and prompts, and work with top companies in the field to benchmark their latest models.

If you’re excited about cutting-edge media models and care deeply about how creators are using them (from casual consumers to advanced professionals), this role offers a front-row seat to the next wave of AI.

What You'll Do

• Product Development for Media Generation AI Benchmarking: Contribute to Media Generation AI product strategy and help execute the roadmap for our media generation evaluation platforms, including image, video, audio, and emerging modalities - including working with engineers on our team to develop features and potentially coding features yourself
• Curate and Develop Media Generation Evaluation Frameworks: Build and maintain comprehensive frameworks, prompt libraries and evaluation techniques that reflect real-world usage patterns to test the capabilities, strengths and weaknesses of media generation models. Our aim is to reflect not just how consumers and professionals are using media generation models today, but how they might be able to use future models in the coming months and years
• Media Generation Operations: Ensure that coverage of media generation models across Artificial Analysis stays up to date by benchmarking new models and providers - both working with our development team and using our tooling yourself (e.g. running Python scripts)
• Creative Direction & Use Case Analysis: Deeply understand how creators are pushing the boundaries with AI media tools, and translate these insights into evaluation frameworks that help creators understand AI capabilities in the future
• Strategic Marketing Support and Community Engagement: Help amplify the impact of our media generation benchmarks, collaborating with our team to maximize impact of our content across platforms and audiences
• Industry Engagement: Work with with leading companies to benchmark their media generation models, understanding their capabilities and helping shape industry standards for how AI media capabilities are evaluated
• Become AI-Native: Embrace an AI-native workflow, using the latest AI tools to generate leverage in a fast-changing industry and maintain our competitive edge in AI benchmarking

What We're Looking For

Required:

• Strong analytical and critical thinking skills
• Excitement about AI and eagerness to work at the forefront of technological innovation
• Demonstrated excellence, both academically and professionally
• Real interest in media generation AI models - especially image and video models

Beyond that, this opportunity could potentially suit a wide range of backgrounds. We believe that the following skills and experience would likely be helpful for success in this role. We’re not expecting any single candidate to tick off the entire list, but these are some things that we think would be valuable:

• Experience working with image and video generation AI models - especially building products leveraging recent models
• Experience working in a product role at a creative software company (Canva, Adobe, Figma) or in a startup environment
• Experience in strategy consulting
• Academic success in a technical degree programme, especially related to AI (e.g. Computer Science, Engineering), or in related technical field (e.g. Physics, Maths)
• Strong writing skills, especially in relation to explaining technical topics to both technical and non-technical audiences
• Visual communication skills (e.g. making slides, visualizing data)
• A strong perspective on AI scaling laws - and specifically their implications for media models
• Publication record in AI/ML conferences or journals (or more broadly any publication record, including non-technical or non-traditional)
• Familiarity with modern AI language models (e.g. developing systems leveraging LLMs as a component)
• Experience working in software engineering or data science
• Proficiency in Python and data analysis libraries (e.g., pandas, numpy, matplotlib); experience with AI/ML frameworks (e.g. PyTorch, Tensor Flow)

Why Artificial Analysis?

• Work with frontier AI technologies and collaborate with companies building the future of AI
• Make a significant impact in a fast-growing startup
• Competitive compensation package (including potential for equity)
• We offer flexible work arrangements, including remote and hybrid options. Our team is currently split between San Francisco, Sydney, Melbourne.

If you're excited about the challenge of building the future of AI benchmarking and analysis, we'd love to hear from you.

• To apply for the role, please email [email protected] with (1) your CV (and LinkedIn URL), (2) a brief note on why you're excited about joining Artificial Analysis, and (3) a two sentence description of your favorite AI paper or industry development from the last 6 months.