Senior Data Engineer – PySpark & Python (Banking Project)

📍 Job Location: Dubai, UAE

🌍 Open to Candidates: India & Dubai

💼 Experience: 5+ years (strong Python & PySpark experience required)

🏦 Domain: Banking / Financial Services

About the Role

We are hiring a Senior Data Engineer – PySpark & Python for one of our Dubai-based banking clients. The role involves designing, building, and optimizing high-performance, scalable data processing and integration solutions within a distributed banking ecosystem. Candidates based in India or Dubai are encouraged to apply.

Key Responsibilities

• Develop and maintain high-quality, scalable Python applications using Python 3.6+.
• Build and optimize large-scale data processing pipelines using PySpark for enterprise banking systems.
• Work with databases such as PostgreSQL, MongoDB, and other relational and NoSQL technologies.
• Design and develop RESTful APIs using frameworks like Flask and Django.
• Integrate and deploy AI/ML models into production-grade data and application workflows.
• Implement solutions involving multiprocessing, concurrency, and load balancing for high-throughput systems.
• Apply strong understanding of modern Python frameworks, asynchronous programming, and concurrency patterns.
• Work closely with DevOps teams to implement and maintain CI/CD pipelines.
• Actively troubleshoot, debug, and resolve issues within distributed services environments.
• Ensure resiliency, scalability, observability, and monitoring of production systems.
• Follow best practices for secure coding, unit testing, documentation, and code reviews.

Required Skills & Qualifications

• Must be currently working on Banking domain.
• Strong hands-on coding experience with Python (3.6 or higher).
• Solid experience with PySpark and distributed data processing frameworks.
• Experience working with PostgreSQL, MongoDB, or similar databases.
• Hands-on experience developing REST APIs using Flask and/or Django.
• Good exposure to AI/ML model integration in real-world applications.
• Strong understanding of multiprocessing, concurrency, and load balancing concepts.
• Experience working in distributed systems and production environments.
• Familiarity with CI/CD principles, automation, and DevOps practices.
• Experience with monitoring, logging, and troubleshooting large-scale systems.

Good to Have

• Prior experience in banking or BFSI projects.
• Exposure to cloud platforms (AWS, Azure, GCP).
• Experience with Docker, Kubernetes, or containerized deployments.
• Knowledge of data security, governance, and compliance standards.