CareerPlan

Software Engineer, Web Crawling

ZurichFull-time2026-10-29 → 2026-11-19
On-siteVisa sponsorship offered

Core

Build massive-scale distributed web crawling infrastructure to process the entire web for an AI search engine.

Role type

Senior IC web crawler engineer

Builds

Distributed crawlers handling 100M+ pages/day, crawl scheduling systems, and anti-bot detection mechanisms

Domain

Internet infrastructure / Web crawling / AI search

Deliverable

production ML models

Required skills

High-performance language (C++, Rust), TypeScript, Playwright, Chrome DevTools Protocol (CDP), distributed systems design, rate limiting, JavaScript rendering, anti-bot detection, crawl scheduling algorithms

Preferred skills

Experience scaling web crawlers to Google-scale, optimizing system performance to exceptional degree

Technologies

C++, Rust, TypeScript, Playwright, Chrome DevTools Protocol