Senior Principal Software Engineer

Level - IC5, Senior Principal Software Engineer

Location - Bangalore, Hybrid

Experience required - Over 15 years, tech stack - Core Java, MLE, AI, Distributed Systems, Rest API's, Microservices

Looking for a strong, hands-on Architect for the Generative AI Service / Agent Platform engineering team at India Development Center. Candidate should be proficient designing distributed systems at scale, with experience working in a cloud environment, and building highly available services. Hands-on experience designing and delivering large-scale cloud infrastructure service or platform services is mandatory. Candidate should be knowledgeable about or have experience in one or more of the following areas – AI models, LLMs, machine learning frameworks, ML engineering / deployment, ML infrastructure, ML Ops, Generative AI.

What OCI Generative AI Cloud Services are:

OCI Generative AI is a managed service o seamlessly integrate LLMs into a wide range of enterprise use cases. The OCI Generative AI service supports Meta’s Llama 2 and Cohere’s models, together with a multilingual embeddings capability for over 100 languages. The service makes it easier to work with LLMs with functionality such as LangChain integration, endpoint management, and content moderation.

OCI Generative AI includes an improved GPU cluster management experience with multi-endpoint support to host clusters, the ability to handle more model requests by scaling clusters, and endpoint analytics. The OCI Generative AI service also offers flexible fine-tuning available for Cohere’s Command 52/6B models. As a result, enterprises can continue to feel confident that their AI precisely fits their specific business context.

With the OCI Generative AI service, customers can tackle a variety of use cases including:

Customer operations: Automating customer service based on a customer’s product suite, experience, and language

Marketing: Mass personalization of search, outreach, and customer nurture based on buyer profile and purchase history.

Sales: Creating virtual sales representatives that guide prospects through offerings during a business transaction

Risk and legal: Accelerating contract writing and drafting based on existing best practices, with multilingual support.

Strategy and finance: At scale monitoring of competitors and updates from customers, across public and private sources.

What is OCI Generative Agent Platform

OCI Generative AI Agents is a fully managed service that combines the power of large language models (LLMs) with an intelligent retrieval system aimed at creating contextually relevant answers by searching your knowledge base.

OCI Generative AI Agents supports several ways to on-board your data where you and your customers can interact with your data using a chat interface or API.

Key Features

• Supports several data on-boarding methods and interaction channels (chat interface or API)
• Creates contextually relevant answers by searching your knowledge base
• Provides source attribution for every answer
• Offers hybrid search capabilities (lexical and semantic)
• Includes content moderation options for input and output
• Supports multi-turn conversations, where users can ask follow-up questions and receive answers that consider the context of previous questions and answers
• Can interpret data from two-axis charts and reference tables in a PDF, without needing explicit descriptions of the visual elements
• All the hyperlinks present in the PDF documents are extracted and displayed as hyperlinks in the chat response.

Your Opportunity:

As we innovate to provide a complete, end-to-end platform for generative AI, with advanced security, best-in-class data management, and a comprehensive portfolio of cloud applications able to address any business problem, we will be extremely happy to have you join us and share the very future of our Generative AI platform by building a cloud native Generative AI service.

Our vision is to provide a world-class Generative AI experience on Oracle Cloud. Aggressive as it might sound, our growth journey is fuelled by highly energetic, technology savvy engineers like YOU who are looking to grow with us to meet the demands of building a powerful next-generation platform. Are you ready to do something big?

Responsibilities

As a Consulting Member of the software engineering division, you will participate in all activities of development and operations of OCI Generative AI Service / Agent Platform.

• Technical Leadership: Provide thought leadership and technical guidance to development and operations teams on best practices, cloud infrastructure, Service Architecture of Generative AI services / Agent platform and enterprise specific services around Generative AI with best-in-class security for customers
• Performance Optimization: Identify performance bottlenecks and apply necessary optimizations to enhance Gen AI Service / Agent platform performance and scalability.
• Security Management: Ensure the security of cloud solutions by designing and implementing secure architectures, performing risk assessments, and adhering to best practices for identity management and data protection.
• Cost Optimization: Monitor and optimize cloud resource like GPUs, Caches, Vector stores usage to ensure cost efficiency while maintaining high performance and scalability. Strategize Deploying and scale Gen AI model on K8s cluster with optimal resources and maximum throughput.
• Compliance and Governance: Ensure compliance with industry standards and organizational policies regarding data privacy, security, and cloud governance.
• Operation Excellence: Troubleshoot, mitigate and resolve performance and design issues and incidents in production carve out runbooks, contribute to runbooks and present in operations review forums, Provide on-call manager OCI Generative AI Services to customers to manage service outage and swiftness to recovery
• Hiring and Enablement : Participate in talent hiring, bring in the best engineering mindset and enable them for success
• Disaster Recovery and High Availability: Design and implement high availability, backup, and disaster recovery strategies for cloud-based systems and applications.
• Collaboration & Documentation: Work closely with other architects, engineers, and stakeholders to develop and maintain system designs, architecture diagrams, and documentation. Work with applied scientists and address external and internal shareholder’s product challenges
• Innovation: Stay current with the latest advancements in cloud technologies, Generative AI, Agentic Platform, Multi Agent System and Tooling and evaluate their applicability to improve infrastructure solutions and increase customer adoption

Qualifications

• Bachelor’s or master’s degree or equivalent experience (preferred) in Computer Science, Computer Engineering, or related technical field.
• 15+ years overall experience in software development, with at least 4 years architecture and designing distributed systems in cloud
• Practical experience in Architecture, design, implementation and production deployment of distributed systems using microservices architecture and API’s using common frameworks like Spring Boot (Java), Vertex.io etc
• Working knowledge of current techniques and approaches in machine learning and statistical or mathematical models both classical and LLMs
• Practical experience working in a cloud environment: Oracle Cloud (OCI), AWS, GCP, Azure
• Strong drive to learn and master new technologies and techniques
• Deep understanding of system architecture, data structures, algorithms, and excellent problem-solving skills.
• Experience with cloud-native technologies, such as Kubernetes, Docker, and serverless computing.