Data Engineer

Position: Data Engineer

Type: Permanent | Hybrid

Location: Sydney, Australia

Eligibility: Must have full work rights in Australia

About the Role

We’re looking for a skilled Data Engineer to build and manage real-time, scalable data pipelines using Kafka, PySpark, and DynamoDB in a cloud-first (AWS) environment. This role will focus on data streaming, ingestion, transformation, and storage solutions across varied industries.

Key Responsibilities

• Design and develop real-time and batch data pipelines
• Implement event-driven architecture using Kafka
• Work with PySpark for large-scale data processing
• Model and optimize data in AWS DynamoDB
• Develop scripts and utilities using Python, SQL, Shell
• Collaborate with cross-functional teams to deliver robust data solutions
• Ensure performance, reliability, and scalability of data flows

Tech Stack & Skills

• Strong hands-on experience with:
• Kafka (data streaming & messaging)
• PySpark (distributed data processing)
• DynamoDB (NoSQL modeling, AWS-native)
• Python, SQL, Shell scripting
• Experience working in AWS environments
• Familiarity with CI/CD, DevOps practices, and monitoring tools
• Excellent problem-solving and communication skills

Nice to Have

• Exposure to AWS services like Lambda, S3, Glue, Redshift
• Experience with data governance and security practices
• Familiarity with tools like Airflow, Terraform, or Docker