We are looking for Data Acquisition Research Engineers. They will develop and maintain large-scale web scraping and data acquisition systems. The role focuses on designing, developing and constantly maintaining resilient data collection and acquisition systems and scrapers for complex web platforms. This includes environments where adversarial and reverse engineering may be required.  

You will develop robust data pipelines and ensure reliable collection of large volumes of structured and unstructured data. 

Responsibilities 

• Develop and maintain data acquisition systems: web scrapers and automated data collection  
• Develop data extraction methods that work with data repositories such as Websites, APIs, and GraphQL endpoints 
• Research and solve data collection challenges in adversarial environments (rate limits, blocking, anti-bot systems, ...) 
• Develop and integrate scalable and reliable data pipelines 
• Monitor scrapers and ensure data quality and pipeline stability 
• Work closely with other teams to transfer collected data into downstream systems 

Requirements 

• Strong Python development experience 
• Experience with web scraping frameworks (Scrapy, Playwright, Selenium, Puppeteer, etc.) 
• Familiarity with GraphQL APIs and REST APIs 
• Solid understanding of networking fundamentals (HTTP/HTTPS, proxies, headers, cookies, rate limiting) 
• Experience handling anti-bot protections and detection mechanisms 
• Experience with CI/CD workflows for testing and deploying scraping infrastructure 
• Ability to parse and process HTML, JSON, and dynamic web content 

Soft Skills 

• Patience and persistence in handling complex data acquisition challenges 
• Competitive, results-driven mindset with a strong drive to solve difficult problems 
• Attention to detail and commitment to data quality 
• Ability to adapt quickly to changing platforms and adversarial environments 
• Good communication and collaboration skills 

Nice to Have 

• Experience building distributed scraping systems 
• Familiarity with containerization and cloud infrastructure 
• Experience with data processing pipelines or large-scale data systems