Data Warehouse Developer

Job Title: P1 Data warehouse Engineer

Location: Brussels, Belgium

Duration: 10 Months (Possible of Extension)

Language: Dutch

Work Mode: Hybrid

Job Description

Description

Demonstrable experience with data warehouse modeling

Demonstrable experience with project-oriented work and data warehouse modeling or demonstrable training in data warehouse modeling

Demonstrable knowledge and experience with planning, designing and building a data warehouse application using tools such as Cognos or Datastage

Demonstrable knowledge and experience of data warehousing concepts, processes and architectures

Demonstrable knowledge and experience of Agile development methodology, in which a leading role can be taken

Demonstrable knowledge and experience with building the following data warehouse components: data integration (ETL), reporting & analysis (BI), databases and metadata

Language requirement: Dutch at European CEFR - level C2

Context.

The Terra platform is set up on Amazon Web Services and uses various PAAS and IAAS solutions. The data lake/data warehouse that is used for all kinds of reporting and analysis purposes uses, among others, the following Amazon Web Services:

Airflow for orchestrating data pipelines

S3 as storage for the data lake

AWS Lambda for the smaller data pipelines

AWS EMR & AWS Batch for the larger data jobs

AWS Redshift as data warehouse, with AWS Athena for querying files on AWS S3

SQL Server on AWS RDS as backend database for the Terra web application

AWS DMS for synchronization between RDS and Redshift

Terraform as Infrastructure as Code to deploy AWS resources on the different environments For example, to process the consumption data that is supplied on a daily basis, Python and Spark are used as an ETL process to merge the new consumption data in the Redshift DWH with existing data. In addition, links are made with numerous other data sources to enrich the data warehouse.

GIT is used for version control of the code and the release process (dev, stg, prod) uses MS Devops. The latter is also used for following up sprints, user stories, tasks, ...

Nice to haves:

Knowledge of Azure Pipelines or a similar CI/CD tool

Knowledge of DBT (data build tool)

Experience with Docker (for AWS Lambda, AWS Batch & AWS EMR)

Specific Skills

Demonstrable experience in Python: (Must have)

Demonstrable experience with Airflow: (Must have)

Demonstrable experience with Data Pipelines: (Must have)

Demonstrable experience with Redshift: (Must have)

Demonstrable experience with Tableau: Expert: 8.00 years - (Must have)

Demonstrable experience with AWS: (Should have)

Demonstrable experience with GIT: (Should have)

Demonstrable experience with SQL Server: (Should have)

Sector Skills

Demonstrable experience in utilities: (Should have)

Demonstrable experience in a government context (Flemish Government, federal, etc.): Nice to have