Senior Data Engineer

Location: Ireland, Hybrid

Employment Type: 12-month Contract

Job Summary

We are seeking a highly skilled Senior Data Engineer to join a dynamic, fast-growing team. The ideal candidate will have strong experience developing and delivering scalable data solutions using Azure Databricks, Azure Data Lake, event streaming technologies, and optionally graph databases such as Azure Cosmos DB or AWS Neptune.

This is a hands-on technical leadership role, providing guidance to junior engineers, driving engineering best practices, and contributing to the evolution of the company’s cloud data platform.

Key Responsibilities

• Design, develop, and optimize scalable data pipelines using Azure Databricks, PySpark, and Azure Data Lake.
• Build and maintain robust ETL/ELT workflows ensuring reliability, data quality, and operational excellence.
• Optimize performance for Spark workloads, graph ETL pipelines, and large-scale distributed data processing.
• Collaborate closely with the Data Engineering Lead/Manager to plan, review, and improve engineering designs.
• Mentor and coach junior/mid-level engineers, fostering strong engineering culture.
• Implement CI/CD pipelines using Azure DevOps for automated deployment and testing.
• Conduct design/code reviews, provide best-practice guidance, and support continuous improvement.
• Integrate data from multiple sources into a centralized platform, ensuring scalability and maintainability.
• Troubleshoot complex data issues and ensure high reliability of pipelines.
• Collaborate effectively with cross-functional teams across multiple locations.

Requirements

• Bachelor’s or Master’s degree in Computer Science, IT, or related field.
• 5+ years of experience in modern data engineering roles.
• 3+ years of hands-on experience with Azure Databricks, Azure Data Lake, Spark, and large-scale data processing.
• Strong programming experience in Python and SQL; familiarity with Scala is a plus.
• Experience with big-data technologies: Spark, Hadoop, Hive, Delta Lake.
• Proficient with Azure cloud services (ADF, Azure DevOps, etc.).
• Solid understanding of ETL/ELT best practices, data modelling, and data warehousing.
• Experience with graph ETL or graph databases (Azure Cosmos DB / AWS Neptune) is advantageous.
• Strong CI/CD and DevOps experience (Git, automated deployments, testing).
• Excellent problem-solving skills and ability to resolve complex data challenges.
• Strong communication and collaboration skills, able to mentor others and contribute to team strategy.

Preferred Qualifications

• Microsoft Certified: Azure Data Engineer Associate.
• Databricks Certified Associate Developer for Apache Spark.
• Knowledge of data governance and security practices.
• Experience with Docker and Kubernetes for containerization and orchestration.