Member of Technical Staff (Search Crawler Analyst)
Core
Design metrics, build data pipelines, and improve the quality of search and answer systems at the intersection of data analysis and engineering.
Role type
Senior IC data analyst / ML engineer (search infrastructure)
Builds
Crawling and storage systems, search and answer systems
Domain
Internet search, web crawling, data infrastructure
Deliverable
production ML models
Required skills
Data analysis, Python, ML model training, metric design, data pipeline engineering, experiment design
Preferred skills
Web crawling, indexing pipelines, LLM-as-a-judge labeling
Technologies
Python, LLMs
Responsibilities
Diagnose quality issues in the crawling pipeline, train small models to optimize pipeline aspects, build datasets for model training, improve page selection algorithms, design and analyze experiments to validate improvements
Seniority
Senior, hands-on IC