Azure Data Engineer

Role: Azure Data Developer

Location: Eindhoven, Netherlands

Role Overview

We are seeking an experienced Azure Data Developer with 8–10 years of experience to design and implement scalable data solutions on Microsoft Azure. The ideal candidate should be capable of independently designing solutions based on high-level architecture and implementing end-to-end data pipelines using Azure Databricks and related Azure services.

This role involves close collaboration with business stakeholders to gather requirements for data pipelines and data lake migration initiatives while ensuring robust monitoring, logging, and operational support.

Key Responsibilities

• Design and develop scalable data solutions based on high-level architectural guidelines.
• Implement data ingestion and processing pipelines from disparate data sources.
• Perform data preprocessing and transformation using Azure Databricks.
• Collaborate with customers to gather business requirements related to data pipelines and data lake migration.
• Develop and manage data pipelines using Azure services.
• Monitor Azure DevOps and Databricks pipelines and participate in support activities.
• Implement serverless architectures using Azure-native services.
• Develop ARM templates for infrastructure provisioning.
• Ensure best practices in logging, monitoring, and performance optimization.
• Continuously learn and implement Azure monitoring and logging services.

Must-Have Technical Skills

• Strong programming skills in Python and Scala
• Experience with SQL and NoSQL databases

Hands-on expertise in:

• PySpark, Spark-SQL
• Azure Databricks
• Azure Data Lake Storage (ADLS)
• Azure Data Factory (ADF)
• Unity Catalog
• Azure Stream Analytics
• Azure SQL Data Warehouse (SQL DW)
• Cosmos DB
• Azure Functions
• Experience in Serverless Architecture
• Infrastructure as Code using ARM Templates
• Monitoring Azure DevOps pipelines and Databricks jobs

Good-to-Have Skills

• Kubernetes and AKS
• Exposure to GCP
• Shell scripting
• DevOps practices and tools
• Git
• Terraform
• Basic understanding of Machine Learning and AI