AWS Glue Data Engineer
Deeplight AIUnited Arab Emirates3 days ago
Full-timeOther
Track This Job
Add this job to your tracking list to:
- Monitor application status and updates
- Change status (Applied, Interview, Offer, etc.)
- Add personal notes and comments
- Set reminders for follow-ups
- Track your entire application journey
Save This Job
Add this job to your saved collection to:
- Access easily from your saved jobs dashboard
- Review job details later without searching again
- Compare with other saved opportunities
- Keep a collection of interesting positions
- Receive notifications about saved jobs before they expire
AI-Powered Job Summary
Get a concise overview of key job requirements, responsibilities, and qualifications in seconds.
Pro Tip: Use this feature to quickly decide if a job matches your skills before reading the full description.
DeepLight AI is a specialist AI and data consultancy with extensive experience implementing intelligent enterprise systems across multiple industries, with particular depth in financial services and banking. Our team combines deep expertise in data science, statistical modeling, AI/ML technologies, workflow automation, and systems integration with a practical understanding of complex business operations.
We are seeking a skilled AWS Glue Data Engineer to join our Data Factory Squad, responsible for migrating source systems into the Lakehouse ingestion zone. This role focuses on building scalable ingestion pipelines, optimizing performance, and ensuring compliance with architectural and data assurance standards.
You will ideally have experience working in financial services with strong experience in AWS Glue, PySpark, and ETL pipeline development.
Your responsibilities as the AWS Glue Data Engineer will include:
You will have experience in:
While technical mastery is the foundation of what we do, the ability to bridge the gap between complex data science and actionable business value is what defines your success with Deeplight.
We're looking for individuals who are not only world-class in their fields of specialism, but also compelling communicators and persuasive advocates for their own skills.
You will be the face of our firm, tasked with building trust, articulating the "why" behind your technical decisions, and effectively "selling" your vision to high-level stakeholders.
If you thrive on the challenge of presenting cutting-edge solutions as much as you do on building them, you will fit right in.
Benefits
Benefits & Growth Opportunities:
At DeepLight AI, we recognise that diversity drives innovation. We are committed to fostering an inclusive environment where individuals with different thinking styles can thrive and contribute their unique strengths to our specialised AI and data solutions.
Our goal is to ensure our application and interview process is accessible, predictable, and fair for all candidates.
If you require any specific adjustments to the application process, or if you require any reasonable adjustments should you be successful in being processed to the interview stage, please do let us know. This information will be kept strictly confidential and will not impact hiring decisions.
We are seeking a skilled AWS Glue Data Engineer to join our Data Factory Squad, responsible for migrating source systems into the Lakehouse ingestion zone. This role focuses on building scalable ingestion pipelines, optimizing performance, and ensuring compliance with architectural and data assurance standards.
You will ideally have experience working in financial services with strong experience in AWS Glue, PySpark, and ETL pipeline development.
Your responsibilities as the AWS Glue Data Engineer will include:
- Data Ingestion Development
- Building and implementing AWS Glue jobs for Bronze layer ingestion using defined standards and templates
- Implementing correct loading methods based on source requirements (CDC, full load, delta, snapshot)
- Designing and executing historical loading mechanisms to bring legacy data into the Lakehouse
- Performance Optimisation
- Optimising Glue job performance (DPU allocation, parallelization, partitioning) according to best practices
- Collaborating with platform teams to ensure tooling and optimization alignment
- Migration & Automation
- Aggressively migrating source tables to Bronze layer, initially using manual approaches with standards/templates, later leveraging AI-enabled acceleration
- Ensuring jobs are version-controlled and production deployment is automated via Git and Terraform
- Governance & Monitoring
- Implementing source system connectivity into CDP in collaboration with source system owners
- Ensuring jobs comply with data contracts and are properly monitored
- Preparing documentation and handover to operational support teams
- Collaboration
- Working closely with Data Architect for ingestion patterns and standards
- Coordinating with Data Assurance Lead to apply quality checks across all jobs
- Partnering with platform engineers for tooling and optimisation
You will have experience in:
- AWS Glue, PySpark, and ETL pipeline development;
- substantial knowledge of Lakehouse architecture and Medallion design principles;
- familiarity with CDC, delta loads, and historical data ingestion strategies; and;
- 5+ years experience in data engineering roles, with hands-on experience in AWS Glue
- AWS services: Glue, S3, Athena, Lambda;
- Git, Terraform for CI/CD automation;
- data quality frameworks (e.g., Soda Core);
- identifying ways to automate their work / repetitive tasks;
- working in a fast-paced environment and deliver aggressive migration targets;
- collaborating and communication with different stakeholder levels; and;
- working with Jira and agile way of working
While technical mastery is the foundation of what we do, the ability to bridge the gap between complex data science and actionable business value is what defines your success with Deeplight.
We're looking for individuals who are not only world-class in their fields of specialism, but also compelling communicators and persuasive advocates for their own skills.
You will be the face of our firm, tasked with building trust, articulating the "why" behind your technical decisions, and effectively "selling" your vision to high-level stakeholders.
If you thrive on the challenge of presenting cutting-edge solutions as much as you do on building them, you will fit right in.
Benefits
Benefits & Growth Opportunities:
- Competitive salary and performance bonuses
- Comprehensive health insurance
- Professional development and certification support
- Opportunity to work on cutting-edge AI projects
- Flexible working arrangements
- Career advancement opportunities in a rapidly growing AI company
At DeepLight AI, we recognise that diversity drives innovation. We are committed to fostering an inclusive environment where individuals with different thinking styles can thrive and contribute their unique strengths to our specialised AI and data solutions.
Our goal is to ensure our application and interview process is accessible, predictable, and fair for all candidates.
If you require any specific adjustments to the application process, or if you require any reasonable adjustments should you be successful in being processed to the interview stage, please do let us know. This information will be kept strictly confidential and will not impact hiring decisions.
Key Skills
Ranked by relevanceReady to apply?
Join Deeplight AI and take your career to the next level!
Application takes less than 5 minutes