Cloud Engineer

Job Summary: We are seeking a seasoned Azure DevOps Engineer with a strong focus on DataOps, data sourcing, data management, and test management pipelines. The ideal candidate will have deep experience in implementing CI/CD pipelines for data platforms, automating data workflows, and ensuring robust data quality and governance. Exposure to Databricks and big data platforms is a strong plus. Key Responsibilities: • Design, implement, and manage CI/CD pipelines for data engineering and analytics projects. • Automate data sourcing, data ingestion, and data transformation processes using Azure DevOps and related tools. • Develop and maintain test management pipelines to ensure data quality, validation, and lineage. • Collaborate with data engineers, architects, and QA teams to streamline DataOps practices. • Integrate and orchestrate workflows across Azure Data Factory, Databricks, Synapse, and other Azure services. • Monitor and troubleshoot pipeline failures, performance bottlenecks, and deployment issues. • Implement infrastructure-as-code (IaC) using tools like Terraform, ARM templates, or Bicep. • Ensure compliance with data governance, security, and privacy standards. • Participate in Agile ceremonies and contribute to continuous improvement initiatives. Required Technical Skills: • 7–9 years of experience in DevOps, with at least 3+ years in Azure DevOps. • Strong understanding of DataOps principles and data lifecycle management. • Experience with Azure DevOps Pipelines, Repos, Artifacts, and Boards. • Proficiency in PowerShell, YAML, and scripting for automation. • Hands-on experience with Azure Data Factory, Azure Synapse, and Azure Storage. • Familiarity with Databricks, Apache Spark, and Delta Lake. • Experience with test automation frameworks for data validation and quality checks. • Knowledge of Git, branching strategies, and release management. Soft Skills: • Strong analytical and problem-solving skills. • Excellent communication and collaboration abilities. • Ability to work independently and in cross-functional Agile teams. • Detail-oriented with a focus on quality and reliability. • Proactive and continuous improvement mindset. Agile & SAFe Ways of Working: • Experience working in Agile or SAFe Agile environments. • Participation in PI Planning, sprint ceremonies, and system demos. • Familiarity with Agile tools like Azure Boards, JIRA, or VersionOne. • Ability to contribute to Agile Release Trains (ARTs) and Program Increments (PIs). Preferred Qualifications: • Certifications in Azure DevOps, Azure Data Engineer, or Databricks. • Experience with Terraform, Bicep, or ARM templates for IaC. • Exposure to data governance frameworks and compliance standards (e.g., GDPR, HIPAA).