Web Scraper
Core
Design, implement, and document robust Scrapy spiders to optimize data collection for digital commerce monitoring, bypassing anti-bot countermeasures.
Role type
Senior Python Developer (Web Scraping)
Builds
Web crawling solutions and data collection pipelines for retail analytics
Domain
Retail Analytics / Digital Commerce
Deliverable
production ML models | product features
Required skills
Python, Scrapy, BeautifulSoup, HTTP protocol, RESTful APIs, GraphQL, Django/FastAPI, Docker, Git, pandas, Linux, bash scripting, cloud providers (AWS/GCP/Azure)
Preferred skills
Time series databases (InfluxDB), single-page application handling
Responsibilities
Design and implement Scrapy spiders resilient to website changes; conduct code review and data validation; architect new developments across codebases; provide technical documentation and training
Seniority
Senior, hands-on IC
