About the Role:

As part of a cross-functional team of engineers, data scientists, and product owners, you will be responsible for designing, implementing, optimizing, and maintaining our machine learning operations (MLOps) infrastructure. If you are passionate about bringing machine learning models from development to production seamlessly and efficiently, Avrioc is the place to be for you!

Responsibilities:

• Build, deploy, and manage AI models and applications in production environments, ensuring scalability and reliability.
• Productionize AI deployments, leveraging Kubernetes, Ray, and LLMOps to run models and services at scale.
• Develop and optimize APIs using FastAPI, enabling smooth integration of AI models into various applications.
• Use tools like Chainlit, Streamlit, and vllm to build and deploy interactive AI applications, including chatbots and intelligent agents.
• Integrate large language models (LLMs) with external APIs (e.g., internet search or other services) to create advanced, multi-functional agents.
• Work with cloud platforms such as AWS and Azure to deploy, monitor, and scale AI applications effectively.
• Collaborate with Data Scientists, AI Engineers, and Software Engineers to design solutions that support both research and production needs.
• Manage and version control code using Git, ensuring efficient collaboration and deployment processes.
• Utilize machine learning frameworks such as PyTorch and TensorFlow to build and fine-tune models for production use cases.
• Contribute to the continuous improvement of the AI/ML pipeline, implementing best practices in model deployment, monitoring, and maintenance.
• Containerization: Implement and manage Docker-based containerization and orchestration using Kubernetes and EKS for deploying large language models (LLMs).
• LLMOps Practices: Apply and implement LLMOps best practices for continuous monitoring of model performance, detecting model drift, managing prompts, and establishing feedback loops for continuous improvement.
• Model Optimization: Utilize techniques such as quantization, distillation, and pruning to optimize LLM models for efficient inference on AWS infrastructure.
• Monitoring and Observability: Develop and maintain comprehensive monitoring and alerting systems to track LLM performance, latency, resource utilization, and identify potential biases.
• Prompt Engineering and Management: Create strategies for prompt engineering and management to enhance LLM outputs, ensuring consistency and safety.
• Collaboration: Collaborate closely with data scientists, researchers, and software engineers to ensure seamless integration and deployment of machine learning models.
• Model Deployment: Ensure that machine learning models are properly versioned and deployed into production, staging, or testing environments automatically.
• Operational Excellence: Set up and fully implement scalable machine learning operations environments. Continuously monitor, optimize, debug, and automate MLOps pipelines for increased quality and efficiency at pipeline, module, and system levels.
• Documentation: Document and track all systems, pipelines, and best practices to maintain a high standard of operations.
• Continuous Learning: Keep abreast of the latest technology trends to drive standard methodologies and stay ahead of the curve.

Technical Skills:

• Proficiency with Kubernetes, Ray, and cloud platforms (AWS, Azure) for scaling AI solutions.
• Hands-on experience with machine learning frameworks such as PyTorch and TensorFlow.
• Strong programming skills in Python and experience with FastAPI.
• Experience with version control systems, especially Git, in a collaborative environment.
• Knowledge of LLMOps, Chainlit, Streamlit, and vllm is required.
• Proven ability to build and deploy large-scale AI applications that involve real-time or high-performance requirements.
• AWS
• Kubernetes
• Python
• PyTorch/TensorFlow
• Slurm
• Ray
• Nvidia DGX
• Kubeflow, MLflow
• Langchain, ChainLit, Streamlit

Requirements:

• Bachelor’s or Master’s degree in Computer Science, AI, Data Science, or a related field.
• 3+ years of experience in AI engineering or similar roles, with a focus on production-level AI deployments.
• Proficiency with Kubernetes, Ray, and cloud platforms (AWS, Azure) for scaling AI solutions.

Avrioc is an equal-opportunity employer and is committed to diversity and inclusion. We encourage candidates from all backgrounds to apply.