Software Engineer

About the Company

At Kaliba, we believe the best engineers are the ones who get to build something truly meaningful. We're currently partnered with a tech company redefining modern data platforms, and they’re looking for a Software / Data Engineer with a passion for big data to join their team. This is a chance to move beyond standard data processing and truly shape a distributed system that will power complex analytics and machine learning at scale.

About the Role

What You’ll Be Building:

• Engineering with Java/Scala: You'll be the primary architect of high-performance, distributed applications, writing clean and robust code that handles massive data volumes. We are on the lookout for someone that has previously worked with Scala, and will be able to quickly hit the ground running.

• Your expertise in Apache Spark will be essential as you design and optimize data pipelines, ensuring maximum efficiency and performance.

• You'll be a core contributor to our client’s Lakehouse architecture, leveraging next-generation open-source tools to combine the flexibility of a data lake with the reliability of a data warehouse.

• Get hands-on with innovative table formats like Apache Iceberg, Delta Lake, and Apache Hudi. You'll also integrate tools like Unity Catalog, Polaris Catalog, and MLflow to manage data governance and machine learning workflows seamlessly.

Responsibilities

• Analyse and design test programs in multiple languages.
• Design and optimize data pipelines using Apache Spark.
• Architect high-performance, distributed applications.
• Contribute to Lakehouse architecture.
• Integrate data governance and machine learning tools.

Qualifications

• A proven track record of working in Java or Scala development (Scala Preferred).
• A deep understanding of core computer science principles—including data structures, caching mechanisms, networking, and database systems—is non-negotiable.
• Experience with Apache Spark internals and at least one of the key open-source table formats (Iceberg, Delta Lake, or Hudi).

Required Skills

• Strong foundation in distributed systems.
• Experience in Scala or Java development (3+ years)
• Knowledge of core computer science principles.
• Hands-on experience with Apache Spark and open-source table formats.

As your partner in this search, Kaliba is here to help you every step of the way. If you meet the criteria and this role sparks your interest, then let’s have a confidential conversation. Apply online to learn more about this exclusive opportunity.