Astra-North Infoteck Inc. ~ Conquering today’s challenges, achieving tomorrow’s vision!
Data Engineer- Python, SQL, ELT, Power BI
Astra-North Infoteck Inc. ~ Conquering today’s challenges, achieving tomorrow’s vision!Canada2 days ago
Full-timeInformation Technology
Experience Required: 8–10 Years

Role Overview

We are looking for a Data Engineer with strong Python and SQL expertise to build reliable data pipelines that transform semi-structured data from Elasticsearch (ES) URLs into clean, analytics-ready datasets.

You will work in a local environment using Python, DBeaver, SQLite/Postgres/Dremio, establish database connections, flatten JSON/Elasticsearch topics into structured tables, and enable downstream reporting in Power BI for business users.

Key Responsibilities

1. Data Ingestion & Transformation


  • Fetch semi-structured data from ES URLs/APIs (e.g., JSON, Elasticsearch topics)

  • Flatten, normalize, and model data into structured analytics tables

  • Build reproducible ETL/ELT workflows using:

    • Python (pandas, requests, SQLAlchemy)

    • SQL-based transformations

  • Automate data extraction and transformation processes



2. Database Engineering


  • Create and maintain schemas in:

    • SQLite

    • PostgreSQL

    • Dremio

  • Configure and manage local DB connections via DBeaver

  • Optimize queries, indexing strategies, and performance tuning

  • Implement:

    • Data partitioning

    • Incremental data loads

    • Performance optimization strategies



3. Data Quality & Governance


  • Establish validation rules, deduplication processes, and anomaly detection

  • Maintain:

    • Dataset versioning

    • Data lineage documentation

    • Metadata and data contracts

  • Ensure secure handling of:

    • Credentials

    • API tokens

    • Endpoints

  • Use Git for version control

  • Maintain code reviews, unit testing, and CI checks

  • Develop technical documentation and runbooks

  • Support ad-hoc business data requests


Required Skills & Experience

Technical Skills


  • Python for Data Engineering

    • pandas

    • NumPy

    • requests

    • SQLAlchemy

    • JSON handling & API integration

  • SQL (Advanced Proficiency)

    • SQLite

    • PostgreSQL

    • Querying via Dremio

  • Data Modeling

    • Dimensional and normalized models

    • Handling nested/semi-structured data

  • Tools

    • DBeaver (Database connections)

    • Power BI (Data preparation for reporting)

  • Pipelines

    • ETL/ELT design

    • Performance optimization

    • Error handling and logging


Core Competencies

  • Strong collaboration and communication skills

  • Ability to translate business requirements into technical solutions

  • Stakeholder engagement from requirements gathering to delivery


Primary Skills

  • Python

  • PostgreSQL

  • Microsoft Power BI

Key Skills

Ranked by relevance
Apply