Python Developer (Web Scraping)

We’re a team of 500+ professionals who develop cutting-edge proxy and web data scraping solutions for thousands of the world’s best known businesses, including Fortune 500 companies.

What’s in store for you:

You’ll be solving complex challenges and maintaining our own infrastructure with 60PB+ monthly data traffic. Here are its scale and maturity in numbers:

6PB+ Ceph storage
60PB+ monthly data traffic through our systems
300k+ service requests/sec processed
500k+ Kafka messages/sec streamed

A word from the team:

We run one of the most advanced and largest scraping and parsing product in the world. We serve hundreds of thousands of requests per second with a very high success rate. Our scrapers and parsers are used by leading e-commerce, market intelligence, and AI industry players making the work challenging and truly global. The team is a blend of different interesting personalities from different walks of life and nationalities. Here you can find people who are experts in gaming, the Korean language, playing “Ilgas kelias” with a guitar, riding bicycles, and other areas.

Your day-to-day:

Design and develop scalable scrapers
Define resilient scraping strategies, unblock websites for scrapingImprove observability in the system
Develop back-end solutions for scraping & parsing problems of various magnitudes
Maintain the current system and develop new features related to scraping & parsing
Check-out our tech stack here: stackshare.io/oxylabs/oxylabs

Your skills & experience:

Experience working with Python
Understanding of computer science, including data structures, algorithms, computability and complexity
Version Control skills using Git
Knowing your way around asynchronous programming would help you a lot
Practical knowledge of any code testing tool

Nice to have:

Knows how to use Dev tools & Network tab
Knows how to write an XPath
Web development knowledge - HTML/JS/CSS
Knowledge of any front-end framework
Experience working with Pydantic, FastAPI, SQLAlchemy, BeautifulSoup, lxml, httpx, Playwright
Knows how to use an API
Has experience working with MySQL, Redis, Docker, Kubernetes, Elasticsearch, Kibana and monitoring tools like Grafana, Prometheus
Has experience in building scalable systems

Salary:

Gross salary: 3200 - 7000 EUR/month. Keep in mind that we are open to discussing a different salary based on your skills and experience

Up for the challenge? Let’s talk!

Python Developer (Web Scraping)

Key Skills

Related Jobs

Web Scraping Engineer - Python/Go

Backend-Focused Full Stack Developer (Node.js, React)

PHP/Go Developer (Mid-Senior)

Related Jobs

Web Scraping Engineer - Python/Go

Backend-Focused Full Stack Developer (Node.js, React)

PHP/Go Developer (Mid-Senior)

Cookie Settings