Member of Technical Staff, Data Flywheel
Core
Build data flywheels, evaluation pipelines, and feedback loops to turn model usage into rigorous evaluations and targeted training data for open AI models.
Role type
Senior IC machine-learning engineer (data flywheel & evaluation)
Builds
Production evaluation pipelines, data ingestion systems, and training data sets
Domain
Generative AI / Large Language Models (LLMs)
Deliverable
production ML models
Required skills
LLM training and evaluation, automated data/evaluation pipelines, reinforcement learning, reward design, software engineering, data curation
Preferred skills
Experience with human and synthetic data strategies, building large-scale ML systems
Technologies
(Not explicitly listed)
Responsibilities
Identify high-value data sources and partnership opportunities; bring new data sources online from scoping to production integration; design and build evaluations, graders, and feedback loops; analyze model performance and failure modes to create targeted datasets; develop human and synthetic data strategies; build infrastructure for data ingestion, inspection, and evaluation at scale
Seniority
Senior, hands-on IC
