Data Engineer

This is us

At Avenga, we believe that human creativity empowers technology that matters. Operating globally, our 6000+ specialists provide a full spectrum of services, including business and tech advisory, enterprise solutions, CX, UX and Ul design, managed services, product development, and software development.

This is the job

We’re building an AI-powered document management and retrieval platform supporting all our GenAI applications. It ingests content from multiple sources (web scraping, uploads, APIs), processes it with AI for classification and enrichment, and offers intelligent search via a web portal and REST APIs.

This is you

• 4+ years of experience in data engineering with cloud platforms
• Strong Python skills with experience in data processing libraries
• Expert knowledge of Cosmos DB, PostgreSQL, and Azure storage solutions
• Experience with document processing, OCR, and content extraction
• Skilled in web scraping and public API integration
• Solid understanding of data governance, quality management, and observability
• Familiar with vector databases and similarity search mechanisms

This is you

• Experience with Azure OpenAI and AI-powered search
• Knowledge of medallion architecture and data tiering strategies
• Familiarity with React/TypeScript frontend frameworks

This is your role

• Develop an ingestion coordinator and a document processing pipeline
• Implement deduplication, validation, and error handling mechanisms
• Design Cosmos DB and PostgreSQL schemas
• Set up Redis caching and optimize AI Search indices
• Define data retention and archival policies
• Build API integrations and structured scraping with error handling
• Automate content ingestion and enrich metadata