ThoughtBot
DevOps Engineer
ThoughtBotLuxembourg18 hours ago
ContractInformation Technology

Key Responsibilities:

  • Kafka Administration & Operations: Deploy, configure, monitor, and maintain Kafka clusters in a high-availability production environment.
  • Performance Optimization: Tune Kafka configurations, partitions, replication, and producers/consumers to ensure efficient message streaming.
  • Infrastructure as Code (IaC): Automate Kafka infrastructure deployment and management using Terraform, Ansible, or similar tools.
  • Monitoring & Incident Management: Implement robust monitoring solutions (e.g., Dynatrace) and troubleshoot performance bottlenecks, latency issues, and failures.
  • Security & Compliance: Ensure secure data transmission, access control, and compliance with security best practices (SSL/TLS, RBAC, Kerberos).
  • CI/CD & Automation: Integrate Kafka with CI/CD pipelines and automate deployment processes to improve efficiency and reliability.
  • Capacity Planning & Scalability: Analyse workloads and plan for horizontal scaling, resource optimization, and failover strategies.
  • Collaboration: Work closely with development teams to support Kafka-based applications and ensure seamless data flow.
  • Training and technical support to end users and other stakeholders.
  • Writing / Updating procedure:
  • To Contribute to the knowledge base.
  • Work together as a team.

Required Skills & Experience:

  • 2+ years of experience in DevOps, Site Reliability Engineering (SRE), or Kafka administration.
  • Hands-on experience with Apache Kafka (setup, tuning, and troubleshooting).
  • Proficiency in scripting (Python, Bash) and automation tools (Terraform, Ansible).
  • Experience with cloud environments (AWS, Azure, or GCP) and Kubernetes-based Kafka deployments.
  • Familiarity with Kafka Connect, KSQL, Schema Registry, Zookeeper.
  • Knowledge of logging and monitoring tools (Dynatrace, ELK, Splunk).
  • Understanding of networking, security, and access control for Kafka clusters.
  • Experience with CI/CD tools (Jenkins, GitLab, ArgoCD).
  • Ability to analyse logs, debug issues, and propose proactive improvements.
  • Excellent problem-solving and communication skills. An ITIL qualification is an asset.
  • Strong communications skills (oral/written) to interact effectively and professionally with both internal and external customers.
  • Ability to work in an international/multicultural environment.
  • Ability to work both independently and with other team members.
  • Knowledge of AWS and Kubernetes.

Nice-to-have:

  • Experience with Confluent Kafka or other managed Kafka solutions.
  • Knowledge of event-driven architectures and stream processing (Flink, Spark, Kafka Streams).
  • Experience with service mesh technologies (Istio, Linkerd) for Kafka networking.

Certifications in Kafka, Kubernetes, or cloud platforms

Key Skills

Ranked by relevance