Azure Data Engineer

Summary: The Azure Data Engineer will be pivotal in shaping the data-driven culture of the organization by designing, developing, and maintaining the data architecture necessary for effective storage, processing, and analysis of data. This role focuses on ensuring data is available, clean, and compliant for various business needs.

Main Responsibilities:

Design, develop, and implement data solutions using Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Event Hub, and other Azure services.
Build and maintain efficient data pipelines and ETL processes to support data integration and analysis.
Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.
Optimize and tune data processes for performance, reliability, and scalability.
Ensure data security and compliance with industry standards and regulations.
Troubleshoot and resolve data-related issues and discrepancies.
Create and maintain documentation for data solutions, including architecture diagrams, data flow diagrams, and technical specifications.
Stay up-to-date with the latest Azure technologies and best practices for data engineering.

Key Requirements:

8+ years of experience as an Azure Data Engineer or similar role.
Strong expertise in Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), and Event Hub.
Proficiency in programming languages such as PySpark, Scala, or SQL.
Experience with data modeling, data warehousing, and big data technologies.
Knowledge of Delta Lake and other data storage formats.
Familiarity with data integration tools and techniques.
Excellent problem-solving skills and attention to detail.
Knowledge of data governance and data quality best practices.
Strong communication and collaboration skills.
Certification in Microsoft Azure Data Engineering or related areas is a plus.
Bachelor’s degree in Computer Science, Information Technology, or a related field.

Nice to Have:

Experience with cloud data architectures and ecosystems.
Familiarity with machine learning concepts and tools.
Previous experience in a large-scale data engineering environment.

Other Details:

Team Structure: Cross-functional collaboration with data scientists and analysts is expected.
Reporting Lines: Reports to the Data Engineering Manager.

Azure Data Engineer

Key Skills

Related Jobs

Engineer - Cloud Security

Backend Software Engineer

Data Scientist

Related Jobs

Engineer - Cloud Security

Backend Software Engineer

Data Scientist

Cookie Settings