Member of Technical Staff (Search Quality Analyst)
Core
Design metrics, build data pipelines, and improve the quality of search and answer systems at the intersection of data analysis and engineering.
Role type
Senior IC search quality analyst (data & ML)
Builds
Search pipelines, answer systems, and model training datasets
Domain
Search technology, Large Language Models (LLM)
Deliverable
production ML models | product features
Required skills
SQL, Python, metric design from scratch, LLM-as-a-judge labeling pipelines, A/B experiment design, search algorithm optimization
Preferred skills
Apache Spark, Databricks, statistics, search product experience
Technologies
SQL, Python, Apache Spark, Databricks, LLMs
Responsibilities
Diagnose quality issues in the search pipeline, design metrics to track search quality, build datasets for model training, improve search snippet and page selection algorithms, design and analyze A/B experiments
Seniority
Senior, hands-on IC