Lead Data Software Engineer

We are seeking a Lead Data Software Engineer to lead the evolution and maturity of data solutions in a dynamic GCP environment.

The role involves implementing orchestration with Airflow, optimizing data ingestion pipelines, and driving the delivery of scalable, high-performance systems.

Responsibilities

• Improve architecture to enhance the scalability and maturity of data solutions
• Implement orchestration for workflows using Airflow for efficient processes
• Build secure REST services in Python 3.x with technologies like FastAPI and asynchronous patterns
• Optimize data pipelines and workflows by leveraging BigQuery's advanced features like partitioning, clustering, and query tuning
• Design high-load systems with a focus on throughput, backpressure, and scalability
• Utilize GCP services such as Cloud Functions, Pub/Sub, and Cloud Storage to build robust data solutions
• Develop end-to-end ETL pipelines with job orchestration frameworks like Airflow
• Drive the design process from RFCs to full operational runbooks with clear documentation and ownership

Requirements

• 7+ years professional software experience; 4+ years of expertise in Python 3.x within production environments
• Knowledge of FastAPI and Pydantic, with proficiency in async patterns and secure REST APIs
• Competency in SQL, data modeling, and hands-on experience with BigQuery (partitioning, clustering, query optimizations)
• Production expertise in GCP services including Cloud Functions, Pub/Sub, Cloud Storage, IAM/Secret Manager, Cloud Build, or equivalent AWS/Azure stacks
• Background in designing high-load systems with solutions for backpressure, idempotency, and scalability
• Experience working on ETL pipelines using tools like Airflow for job orchestration
• Showcase of excellent communication, ownership, and problem-solving skills

Nice to have

• Familiarity with parsing file formats at scale like CSV, Excel, or XML along with schema evolution/versioning techniques
• Understanding of data quality frameworks such as assertions or expectations and tools like dbt or Dataform
• Background in event-driven architectures or streaming technologies; experience with Beam/Dataflow is a plus
• Expertise in concurrency and performance profiling in Python using asyncio or multiprocessing
• Knowledge of IaC practices using Terraform or SRE methodologies
• Awareness of security concepts such as authentication/authorization, service accounts, and principles of least privilege

We offer

• With us you can:
  • Work on a flexible schedule remotely or from any of our comfortable offices or coworking spaces in Ukraine
  • Receive the necessary equipment to perform your work tasks
  • Change projects and technology stacks within EPAM
  • Gain experience in various business domains (Insurance, E-commerce, Healthcare, Finance, Travelling, Media, Artificial Intelligence, and more)
  • Relocation opportunities may be available for eligible candidates, depending on the role and openings at other EPAM locations
  • Participate in volunteer, charity programs and communities (both technical and interest-based)
• We focus on your professional growth:
  • You can plan your individual career path together with your manager
  • Receive regular feedback from colleagues
  • Improve your English for free with certified teachers (Speaking Clubs, client interview preparation courses, etc.)
  • Get the opportunity to undergo free training and certification in AWS, GCP, or Azure Clouds
  • Use the internal E-learn training program (18,200+ specialized training and mentoring programs)
  • Access corporate accounts on LinkedIn Learning, Get Abstract and other partner resources
  • Study at EPAM Solution Architecture School with the instructors who are practicing architects
  • Develop as a leader, join Delivery Management, Resource Management, Leadership Essentials school and more
  • Participate in internal communities (500+ meetups, technical discussions, brainstorming sessions, online events and conferences annually)
• What we offer:
  • Vacation and sick leave (including a sick leave without a medical certificate)
  • A wide range of Voluntary Medical Insurance programs providing both medical treatment and various preventive options (including sports activities)
  • Medical insurance for family members at corporate rates
  • Company support during significant life events (childbirth or adoption, marriage, etc.)
  • Support for psychological comfort: discounts on services from mental health specialists or coaches, thematic training
  • E-kids program - a free programming language training program for EPAMers' children

Kindly note that this role supports remote work, but only from within Ukraine.

Kindly be advised that the set of benefits, including learning, certification, and other opportunities, may vary depending on the role you apply for. Our recruiter will be able to share more details about the specific opportunity during your general interview.

EPAM strives to provide its global team of over 61,700 professionals in more than 55 countries with opportunities for professional growth from day one of collaboration. Our colleagues are the source of EPAM's success, so we value cooperation, strive to always understand our clients' business and aim for the highest quality standards. No matter where you are, you will join a dedicated, diverse community that will help you realize your potential to the fullest.