Member of Technical Staff, ML Platform (Evals)
Core
Design and build an end-to-end ML evaluation platform to generate samples, score them with automated metrics and human annotations, and track results across checkpoints to enable ship/no-ship decisions for frontier models.
Role type
Senior IC ML Platform Engineer (Evals)
Builds
A unified evaluation platform for generating, annotating, and reviewing model outputs at scale
Domain
Generative AI, World Models, Robotics, Video/Image/Audio
Deliverable
production ML models
Required skills
Python, PyTorch, Kubernetes, GPU workload orchestration, Data pipeline design, Experimental statistics, Systems architecture, Technical leadership
Preferred skills
LLM/VLM-as-judge pipelines, Online experimentation platforms, Agent/robotics policy evaluation
Technologies
Python, PyTorch, Kubernetes, Cloud services
Responsibilities
Own the evaluation platform end-to-end; Define CLIs, APIs, GUIs, and storage layers; Work with research teams to understand needs and generalize into the platform; Set standards for reproducibility and metric definitions; Support platform adoption across training and production; Contribute to broader ML Platform tools
Seniority
Senior, hands-on IC with technical leadership