AI Engineer
Resource CornerAustralia1 day ago
Full-timeEngineering, Information Technology
Track This Job
Add this job to your tracking list to:
- Monitor application status and updates
- Change status (Applied, Interview, Offer, etc.)
- Add personal notes and comments
- Set reminders for follow-ups
- Track your entire application journey
Save This Job
Add this job to your saved collection to:
- Access easily from your saved jobs dashboard
- Review job details later without searching again
- Compare with other saved opportunities
- Keep a collection of interesting positions
- Receive notifications about saved jobs before they expire
AI-Powered Job Summary
Get a concise overview of key job requirements, responsibilities, and qualifications in seconds.
Pro Tip: Use this feature to quickly decide if a job matches your skills before reading the full description.
Location: Sydney/Melbourne/AU offices
Job Description: We are looking for a hands-on and technically strong Senior software engineer to drive the development, and deployment of evaluation frameworks for Agentic AI and Retrieval-Augmented Generation (RAG) solutions. This role is pivotal in enabling the deployment of scalable multi-agent AI architecture (20–30+ agents) in production, with a strong emphasis on Agent-Level and System-Level evaluation in compliance-heavy domains such as finance and banking. Responsibilities: ∙Develop and implement framework leveraging advanced AI Agents using models such as OpenAI GPT, Anthropic Clude, Meta Llama2, Google Gemini etc. focusing on enhancing developer and business productivity. ∙Knowledge of mapping suitable LLMs according to agent roles (e.g., DevSecOps Agent with OpenAI). ∙Must have experience with Agentic AI frameworks like LangGraph, CrewAI, AutoGen etc. ∙Must have experience in framework evaluation tools and techniques such as AgentBench, RAGA and OpenAI Evals. ∙Develop the solution incorporating advanced techniques like Retrieval Augmented Generation (RAG), Prompt Engineering ensuring optimal performance and scalability. ∙Must know how to measure and benchmark the performance of the Agents. ∙Drive end-to-end implementation and deployment with extensive knowledge of Azure and AWS services. ∙Stay abreast of emerging trends, complex patterns, data dependencies, and advancements in AI architecture, contributing to the refinement and innovation of application development processes. Qualifications:
Job Description: We are looking for a hands-on and technically strong Senior software engineer to drive the development, and deployment of evaluation frameworks for Agentic AI and Retrieval-Augmented Generation (RAG) solutions. This role is pivotal in enabling the deployment of scalable multi-agent AI architecture (20–30+ agents) in production, with a strong emphasis on Agent-Level and System-Level evaluation in compliance-heavy domains such as finance and banking. Responsibilities: ∙Develop and implement framework leveraging advanced AI Agents using models such as OpenAI GPT, Anthropic Clude, Meta Llama2, Google Gemini etc. focusing on enhancing developer and business productivity. ∙Knowledge of mapping suitable LLMs according to agent roles (e.g., DevSecOps Agent with OpenAI). ∙Must have experience with Agentic AI frameworks like LangGraph, CrewAI, AutoGen etc. ∙Must have experience in framework evaluation tools and techniques such as AgentBench, RAGA and OpenAI Evals. ∙Develop the solution incorporating advanced techniques like Retrieval Augmented Generation (RAG), Prompt Engineering ensuring optimal performance and scalability. ∙Must know how to measure and benchmark the performance of the Agents. ∙Drive end-to-end implementation and deployment with extensive knowledge of Azure and AWS services. ∙Stay abreast of emerging trends, complex patterns, data dependencies, and advancements in AI architecture, contributing to the refinement and innovation of application development processes. Qualifications:
- Bachelor’s degree or master’s degree (or equivalent experience) or PhD in Computer Science, Electrical Engineering, Artificial Intelligence, or Applied Math with 3+ years of experience, with at least 1 year of hand-on in GenAI.
- Excellent programming skills in Python with strong fundamentals in programming, optimizations and software design.
- Proven expertise in developing scalable evaluation systems for GenAI, RAG, or multi-agent architectures.
- Experience with agentic AI frameworks (e.g., LangChain, CrewAI, AutoGPT), RAG pipelines and orchestration tools.
Key Skills
Ranked by relevanceReady to apply?
Join Resource Corner and take your career to the next level!
Application takes less than 5 minutes