AI Engineer

Location: Sydney/Melbourne/AU offices

Job Description: We are looking for a hands-on and technically strong Senior software engineer to drive the development, and deployment of evaluation frameworks for Agentic AI and Retrieval-Augmented Generation (RAG) solutions. This role is pivotal in enabling the deployment of scalable multi-agent AI architecture (20–30+ agents) in production, with a strong emphasis on Agent-Level and System-Level evaluation in compliance-heavy domains such as finance and banking. Responsibilities: ∙Develop and implement framework leveraging advanced AI Agents using models such as OpenAI GPT, Anthropic Clude, Meta Llama2, Google Gemini etc. focusing on enhancing developer and business productivity. ∙Knowledge of mapping suitable LLMs according to agent roles (e.g., DevSecOps Agent with OpenAI). ∙Must have experience with Agentic AI frameworks like LangGraph, CrewAI, AutoGen etc. ∙Must have experience in framework evaluation tools and techniques such as AgentBench, RAGA and OpenAI Evals. ∙Develop the solution incorporating advanced techniques like Retrieval Augmented Generation (RAG), Prompt Engineering ensuring optimal performance and scalability. ∙Must know how to measure and benchmark the performance of the Agents. ∙Drive end-to-end implementation and deployment with extensive knowledge of Azure and AWS services. ∙Stay abreast of emerging trends, complex patterns, data dependencies, and advancements in AI architecture, contributing to the refinement and innovation of application development processes. Qualifications:

• Bachelor’s degree or master’s degree (or equivalent experience) or PhD in Computer Science, Electrical Engineering, Artificial Intelligence, or Applied Math with 3+ years of experience, with at least 1 year of hand-on in GenAI.
• Excellent programming skills in Python with strong fundamentals in programming, optimizations and software design.
• Proven expertise in developing scalable evaluation systems for GenAI, RAG, or multi-agent architectures.
• Experience with agentic AI frameworks (e.g., LangChain, CrewAI, AutoGPT), RAG pipelines and orchestration tools.