AI Platform Engineer, Applied AI

This position is posted by Jobgether on behalf of a partner company. We are currently looking for an AI Platform Engineer, Applied AI in Australia.

We are seeking a highly skilled AI Platform Engineer to design and implement infrastructure that enables large-scale AI systems to operate reliably and efficiently. In this role, you will focus on production AI systems, building evaluation frameworks, observability tooling, and diagnostic infrastructure to measure and improve system performance. You will collaborate with engineers across the organization to identify gaps, prototype solutions, and implement improvements that enhance AI-driven features. The ideal candidate thrives in a fast-paced, fully remote environment, is comfortable navigating ambiguity, and has a passion for optimizing AI systems that impact millions of users. This is a high-ownership role with significant influence on how AI features are monitored, evaluated, and improved.

Accountabilities

• Build evaluation infrastructure to measure AI system speed, accuracy, and reliability, both offline and in production.
• Develop observability tools and dashboards that surface quality metrics and track improvements over time.
• Diagnose performance gaps across retrieval, ranking, prompting, and other AI workflows.
• Prototype and validate improvements to AI pipelines, including RAG pipelines, chunking strategies, retrieval methods, and re-ranking approaches.
• Analyze usage data to identify opportunities for enhancement and new AI features.
• Collaborate with engineers to ensure production changes improve system quality and reliability.
• Stay current with cutting-edge AI research, techniques, and tools to inform platform improvements.
  
  

Requirements

• Proven experience building and evaluating AI systems in production, including RAG pipelines, retrieval/search systems, and LLM-powered applications.
• Strong proficiency in Python for prototyping and experimentation.
• Willingness to learn and work with Ruby on Rails for integrating with production systems.
• Expertise in AI infrastructure, evaluation pipelines, observability tooling, and data processing.
• Deep understanding of RAG architecture, embeddings, retrieval, re-ranking, and context management.
• Strong experimentation and data analysis skills, including designing A/B tests, measuring results, and iterating quickly.
• Ability to work autonomously in a fast-paced, dynamic environment while collaborating effectively with a team.
• Strong alignment with company values and CEFR Level C2 / ILR Level 5 proficiency in English.
• Bonus: Experience with AI evaluation frameworks such as Braintrust or LangSmith.
  
  

Benefits

• Competitive U.S.-benchmarked salary range: $130,000 – $140,000 USD per year, with annual reviews and potential equity and bonus opportunities.
• Fully remote work with autonomy and focus on outcomes.
• 35 days of PTO annually, plus paid sabbatical after 5 years.
• Comprehensive medical coverage for employees and families or reimbursement options.
• Parental leave for expanding families.
• Home office stipend and professional learning & development support.
• Annual company retreats at international locations.
• Opportunity to work on high-impact AI infrastructure that shapes the user experience for millions of customers.
  
  

Why Apply Through Jobgether?

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Why Apply Through Jobgether?

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.