Data Engineer- Python, SQL, ELT, Power BI

We are looking for a Data Engineer with strong Python and SQL expertise to build reliable data pipelines that transform semi-structured data from Elasticsearch (ES) URLs into clean, analytics-ready datasets.

You will work in a local environment using Python, DBeaver, SQLite/Postgres/Dremio, establish database connections, flatten JSON/Elasticsearch topics into structured tables, and enable downstream reporting in Power BI for business users.

• Fetch semi-structured data from ES URLs/APIs (e.g., JSON, Elasticsearch topics)
  

• Flatten, normalize, and model data into structured analytics tables
  

• Build reproducible ETL/ELT workflows using:
  

  • Python (pandas, requests, SQLAlchemy)
    

  • SQL-based transformations
    

• Automate data extraction and transformation processes
  

  
  
  

• Create and maintain schemas in:
  

  • SQLite
    

  • PostgreSQL
    

  • Dremio
    

• Configure and manage local DB connections via DBeaver
  

• Optimize queries, indexing strategies, and performance tuning
  

• Implement:
  

  • Data partitioning
    

  • Incremental data loads
    

  • Performance optimization strategies
    

  
  
  

• Establish validation rules, deduplication processes, and anomaly detection
  

• Maintain:
  

  • Dataset versioning
    

  • Data lineage documentation
    

  • Metadata and data contracts
    

• Ensure secure handling of:
  

  • Credentials
    

  • API tokens
    

  • Endpoints
    

• Use Git for version control
  

• Maintain code reviews, unit testing, and CI checks
  

• Develop technical documentation and runbooks
  

• Support ad-hoc business data requests
  

  
  

• Python for Data Engineering
  

  • pandas
    

  • NumPy
    

  • requests
    

  • SQLAlchemy
    

  • JSON handling & API integration
    

• SQL (Advanced Proficiency)
  

  • SQLite
    

  • PostgreSQL
    

  • Querying via Dremio
    

• Data Modeling
  

  • Dimensional and normalized models
    

  • Handling nested/semi-structured data
    

• Tools
  

  • DBeaver (Database connections)
    

  • Power BI (Data preparation for reporting)
    

• Pipelines
  

  • ETL/ELT design
    

  • Performance optimization
    

  • Error handling and logging
    

  
  

• Strong collaboration and communication skills
  

• Ability to translate business requirements into technical solutions
  

• Stakeholder engagement from requirements gathering to delivery
  

  
  

• Python
  

• PostgreSQL
  

• Microsoft Power BI