Web Scraping Lead - Global Team (India)
Core
Lead a web scraping engineering team to design, build, and maintain scalable infrastructure for extracting alternative data from dynamic websites.
Role type
Senior engineering manager (web scraping)
Builds
Scalable, reliable web scraping infrastructure and data pipelines
Domain
Market research, alternative data, web automation
Deliverable
production ML models | infrastructure
Required skills
Web scraping architecture, Playwright, Puppeteer, Selenium, browser automation, fingerprinting, proxy rotation, HTTP/REST APIs, monitoring, logging, team leadership, technical roadmap planning
Preferred skills
Engineering management experience, anti-bot challenge solving, cross-functional collaboration
Technologies
Playwright, Puppeteer, Selenium
Responsibilities
Lead and mentor web scraping engineers; own architecture and scalability of scraping systems; guide development of scrapers; solve complex dynamic website challenges; establish monitoring and data-quality practices; partner with cross-functional teams; participate in hiring and technical reviews
Seniority
Senior, hands-on IC with management responsibilities