Senior Data Quality Engineer

We are seeking a knowledgeable Senior Data Quality Engineer to join our team.

In this role, you will be responsible for ensuring the precision and reliability of our data engineering processes and platforms. You will work closely with multiple teams to maintain high standards of data quality across all systems.

Responsibilities

• Design and deploy automated testing frameworks and strategies for data engineering processes
• Integrate with technologies such as Databricks, PySpark, Scala, and Spark SQL
• Build automated tests to verify data accuracy, consistency, and reliability throughout platforms
• Incorporate automated testing into Azure DevOps CI/CD pipelines for streamlined testing, building, and deployment
• Conduct performance testing to evaluate the scalability and efficiency of data processing solutions
• Collaborate with data engineers and stakeholders to gather requirements and align testing strategies with business needs
• Set up monitoring for automated tests and provide ongoing reports on test coverage, defects, and data quality issues

Requirements

• At least 3 years of experience in Data Quality Engineering or a related field
• Thorough understanding of automated testing frameworks and data engineering concepts
• Practical experience with Databricks, PySpark, Scala, and Spark SQL
• Knowledge of Terraform and Azure Event Hubs
• Ability to integrate testing frameworks with Azure DevOps and manage source control using Git
• Experience with pipeline configuration and deployment using Terraform
• Familiarity with Jira for task and issue management
• Strong analytical and troubleshooting skills for complex data and testing problems
• Effective English communication skills, both written and spoken, at B2+ level or higher

Nice to have

• Background with Power BI for data visualization
• Experience with Azure Data Lake and Spark Streaming
• Exposure to large-scale or data-intensive industries
• Ability to implement Infrastructure as Code (IaC) solutions

We offer

• International projects with top brands
• Work with global teams of highly skilled, diverse peers
• Healthcare benefits
• Employee financial programs
• Paid time off and sick leave
• Upskilling, reskilling and certification courses
• Unlimited access to the LinkedIn Learning library and 22,000+ courses
• Global career opportunities
• Volunteer and community involvement opportunities
• EPAM Employee Groups
• Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn