Platform Engineer, RL & Eval Data
Core
Design, build, and maintain distributed platforms for reinforcement learning (RL), model evaluation, and human-data annotation workflows for LLMs and agents.
Role type
Senior Platform Engineer (RL & Eval Data)
Builds
Evaluation harnesses, RL environments, annotation tools, and production data pipelines for LLM training and agents.
Domain
Artificial Intelligence / Machine Learning / Data Infrastructure
Deliverable
production ML models
Required skills
Python, TypeScript, React, Postgres, distributed systems, container orchestration, observability, RL environment design, LLM evaluation metrics, human-data scaling
Preferred skills
Open-source contributions to RL/eval frameworks, building tools for large distributed contributor workforces
Technologies
Python, TypeScript, React, Postgres, Kubernetes, Cloud providers
Responsibilities
Maintain and improve evaluation, RL, and annotation platforms; Design and ship features across databases, backend services, and APIs; Scale distributed systems for larger workloads; Build RL environments and infrastructure for sandboxes and reward computation; Operate reliable production systems with security and monitoring; Build tooling for task authors and reviewers; Collect high-throughput rollouts; Build evaluation harnesses and reward functions for LLMs.
Seniority
Senior, hands-on IC