Senior DevOps Engineer

We are seeking a hands-on Senior DevOps Engineer to join our Data and Analytics Platform team, driving the design and implementation of our next-generation cloud-based, streaming Data Mesh platform. This role is pivotal for enabling teams to create scalable, data-driven products and advancing our application modernization strategy.

Design and Implement Cloud Infrastructure: Architect and deploy scalable cloud infrastructure using Infrastructure as Code (IaC) principles to support our Data Mesh platform. Experience with major cloud providers (AWS/Azure) and modern IaC tools (Terraform, CloudFormation) is essential.

Build Automation Pipelines: Develop and maintain CI/CD pipelines for data products, ensuring reliable deployment, testing, and monitoring of data infrastructure and applications. Experience with tools like Jenkins, GitLab CI, or GitHub Actions is required.

Implement Platform Observability: Design and deploy comprehensive monitoring, logging, and alerting solutions across the data platform. Experience with tools like Datadog, Prometheus, Grafana, ELK stack, or similar technologies is crucial.

Enable Self-Service Infrastructure:

Develop automated provisioning systems for data infrastructure
Create and maintain infrastructure templates and patterns
Implement GitOps workflows for infrastructure management
Build developer portals and self-service tooling

Security and Compliance Automation:

Implement security controls and compliance checks as code
Automate security scanning and vulnerability management
Design and implement IAM and access management solutions
Maintain audit trails and compliance reporting automation

Performance Optimization:

Monitor and optimize platform performance
Implement auto-scaling solutions design for high availability and disaster recovery
Optimize cost through infrastructure efficiency

Lead Platform Re-Architecture:

Transform our existing data platform architecture to align with Data Mesh principles
Design and implement our Apache Iceberg adoption strategy, enabling ACID transactions and time travel capabilities
Create standardized patterns for teams to effectively adopt and optimize Iceberg tables

Drive Release Excellence:

Establish automated release processes that balance agility with platform stability
Build deployment frameworks with comprehensive validation, monitoring, and rollback capabilities
Implement coordinated release strategies across distributed domains

Enable Technical Success:

Design reference architectures and implementation patterns for domain teams
Lead technical design reviews and architecture governance
Provide technical guidance during the transition to Data Mesh architecture

What We’re Looking For

8+ years of experience in DevOps/Platform Engineering roles within a data operations team

Cloud & Infrastructure:

Extensive experience with AWS, particularly EKS and EKS Anywhere
Deep expertise in Kubernetes and container orchestration
Strong background in Infrastructure as Code using Terraform and Terragrunt
Experience with GitOps practices using ArgoCD
Knowledge of distributed systems and event-driven architectures
Strong knowledge of logging, monitoring, APM tracing and alerting tools (ELK, Datadog, Prometheus, Grafana, etc.)

Development & Engineering:

Proficient in Java development and JVM optimization
Experience with build tools (Maven/Gradle) and dependency management
Strong background in unit testing frameworks and testing practices
Expertise in Git version control and collaborative development
Experience with Apache Kafka and Kafka Streams
Background in monitoring and performance optimization for Java applications

CI/CD & Automation:

Experience designing and managing Jenkins/Github/Gitlab pipelines
Strong scripting skills (Python, Bash)
Understanding of DevOps practices and automation principles

Preferred Qualifications:

Experience with data mesh or similar distributed data architectures
Relevant cloud certifications
Experience building developer platforms or portals
Background in DataOps practices

Senior DevOps Engineer

Key Skills

Related Jobs