Data Engineer

We are HCLTech, one of the fastest-growing large tech companies in the world and home to 225,000+ people across 60 countries, supercharging progress through industry-leading capabilities centered around Digital, Engineering and Cloud. The driving force behind that work, our people, are diverse, creative, and passionate, raising the bar for excellence on a regular basis. We, in turn, work hard to bring out the best in them as we strive to help them find their spark and become the best version of themselves that they can be. If all this sounds like an environment you’ll thrive in, then you’re in the right place. Join us on our journey in advancing the technological world through innovation and creativity.

We are looking for a Data Engineer with Azure and PySpark to join us!

Key Responsibilities:

1. Python/PySpark Expertise:

o Proficiency in Python/PySpark frameworks/libraries for data processing and analysis.

2. Azure Databricks (ADB) Expertise:

o Strong knowledge and experience working with Azure Databricks for data engineering tasks.

3. SQL Skills:

o Expertise in SQL for querying and manipulating data in relational databases.

4. ETL and Process Flow Design:

o Design and implement complex ETL (Extract, Transform, Load) processes and workflows to ensure efficient data movement and transformation.

5. Data Transformation Frameworks:

o Proficiency across various data transformation frameworks to handle diverse data processing requirements.

6. Understanding of Underlying Systems:

o Familiarity with the underlying systems including storage, processing, and visualization technologies to optimize data workflows.

7. Airflow Configuration and Orchestration:

o Configure and orchestrate data pipelines using Apache Airflow for workflow automation and scheduling.

Good to Have:

1. Azure Data Factory (ADF) Expertise:

o Experience with Azure Data Factory for building, orchestrating, and monitoring data pipelines.

2. Database Execution Plans:

o Ability to read and understand database execution plans to optimize query performance.

3. Azure/Azure DevOps Knowledge:

o Familiarity with Azure cloud services and Azure DevOps for managing and deploying data solutions.

4. Configuration Management, Continuous Integration, Continuous Deployment (CI/CD), and Cloud Implementations on Azure:

o Understanding of configuration management practices, CI/CD pipelines, and cloud implementations on the Azure platform.

5. Azure Certification:

o Azure certification is a plus, demonstrating proficiency in Azure cloud technologies and services.

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or related field.

• Proven experience as a Data Engineer or similar role.

• Strong proficiency in Python/pyspark, SQL, and Azure Data Factory.

• Experience designing and implementing ETL processes and data pipelines.

• Familiarity with data transformation frameworks and workflow orchestration tools like Apache Airflow.

• Understanding of database systems, cloud platforms, and CI/CD practices.

• Azure certification.

HCLTech is committed to protecting and securing the privacy and confidentiality of the Personal Data which it collects directly or indirectly from you when applying for a job at HCLTech either directly or through a third-party human resources agency. This notice (the “Notice”) outlines and explains how HCL Technologies Limited including its subsidiaries, local employing entities, associates, and affiliated companies [collectively referred to as “HCLTech”, “us,” “our”, or “we”] will process your Personal Data in accordance with applicable privacy legislation(s).

https://www.hcltech.com/candidate-privacy-notice