Data Engineer/Data Catalog and Quality Analyst

Introduction

A career in IBM Consulting is rooted by long-term relationships and close collaboration with clients across the globe. You'll work with visionaries across multiple industries to improve the hybrid cloud and AI journey for the most innovative and valuable companies in the world. Your ability to accelerate impact and make meaningful change for your clients is enabled by our strategic partner ecosystem and our robust technology platforms across the IBM portfolio; including Software and Red Hat. Curiosity and a constant quest for knowledge serve as the foundation to success in IBM Consulting. In your role, you'll be encouraged to challenge the norm, investigate ideas outside of your role, and come up with creative solutions resulting in ground breaking impact for a wide network of clients. Our culture of evolution and empathy centers on long-term career growth and development opportunities in an environment that embraces your unique skills and experience.

Your Role And Responsibilities

Role Overview: Build, test, and maintain the data pipelines required for effective data ingestion and integration into the AWS datalake.

Key Responsibilities

• Develop and deploy data ingestion processes for structured and unstructured data.
• Implement best practices for data quality and error handling.
• Work with the Data Architect to optimize performance and scalability.
• Ensure compliance with governance and security protocols.
  
  

Preferred Education

Bachelor's Degree

Required Technical And Professional Expertise

• 5-8 years of hands-on experience in data engineering.
• Proficiency in AWS data services (AWS Glue, AWS S3, and optionally AWS Lake Formation).
• Strong programming skills in languages such as Python or Java.
• Experience in ETL/ELT processes and data quality assurance.
  
  

Software Experience

• AWS Glue, AWS S3
• Familiarity with AWS Lake Formation is a plus
• Knowledge of data engineering from an analytics, ML, data science use case and data need fulfillment is necessary.
• Experience is building reusable templates and assets on AWS is essential.