Member of Technical Staff
Core
Build and own end-to-end evaluation suites for LLM capabilities, including task specifications, grading logic, and researcher dashboards.
Role type
Senior IC machine-learning engineer (LLM evaluation & infrastructure)
Builds
Production evaluation harnesses, data pipelines, and RL training environments
Domain
Artificial Intelligence / Large Language Models
Deliverable
production ML models
Required skills
Python, distributed systems, LLM training, evaluation infrastructure, supervised fine-tuning, reinforcement learning, testing, reproducibility
Preferred skills
agentic evaluation, tool use, code execution, sandbox environments, multi-turn tasks, training frameworks, inference stacks, data-labeling operations, human-preference collection, open-source evaluation frameworks, public benchmarks, verifiers, simulation environments
Technologies
Python, distributed systems, RL training environments, reward functions, simulators, verifiers, task generators
Responsibilities
Build and maintain RL training environments including reward functions and simulators; Operate evaluation harnesses as production services with sandboxed execution; Own data pipelines for collection, filtering, deduplication, labeling, and synthetic data generation; Measure evaluation regressions, environment health, throughput, and cost; Collaborate with researchers to turn hypotheses into runnable evaluations and environments
Seniority
Senior, hands-on IC
