Member of Technical Staff (Backend Software Engineer, Search Crawler)
Core
Design, develop, and operate large-scale web crawlers, ingestion pipelines, and data processing systems to support advanced search technologies.
Role type
Senior IC backend software engineer (search crawler)
Builds
Web-scale data ingestion pipelines, crawler services, and data management dashboards
Domain
Internet search infrastructure, distributed systems
Deliverable
production ML models | product features | infrastructure
Required skills
Python, C++, Rust, Go, data structures and algorithms, Kubernetes, AWS, distributed systems architecture, high-load system optimization
Preferred skills
Experience with large-scale web crawlers, hardware-adjacent services
Technologies
Kubernetes, AWS
Responsibilities
Lead projects for developing large-scale web crawlers and ingestion pipelines; Build and optimize core backend and frontend components for crawler services; Collaborate with Search and Infrastructure teams on data ingestion; Architect and implement fullstack features for scalable distributed systems; Iterate and enhance system performance and reliability
Seniority
Senior, hands-on IC
