Machine Learning Engineer - Large Scale Interface

As a Machine Learning Engineer on the ML Platform team, you will play a key role in developing

scalable inference platform powered by LLMs and other generative models. Your focus will be on building infrastructure for both real-time and batch inference, ensuring scalability, extensibility, and enabling personalized solutions at scale for platform users.

Key Responsibilities:

• Develop Cutting-Edge AI Infrastructure: Collaborate with the ML Platform team to design and implement infrastructure for large-scale batch and real-time inference using state-of-the-art generative AI models.
• Ensure Observability and Cost Efficiency: Integrate tools for monitoring system performance, managing queues, and tracking platform costs.
• Support Customization Needs: Build features to accommodate diverse models, fine-tuning requirements, and varied workload priorities.

Required Skills and Qualifications:

• Strong understanding of ML inference with LLMs and generative models, particularly optimizing for performance.
• Proficiency in Python and familiarity with the ML ecosystem.
• Expertise in developing and deploying systems on Kubernetes, with experience in GitOps solutions like ArgoCD being a plus. Knowledge of Helm and Kustomize is also highly regarded.
• Solid DevOps experience, including Infrastructure as Code (IaC) tools like Terraform.
• Excellent written and verbal communication skills.