CareerPlan

Research Engineer - Data and Evaluation

San FranciscoRemoteFull-time$150k–$350k2026-10-16 → 2026-11-19
HybridVisa sponsorship offered

Core

Design and build challenging evaluations, executable environments, and RL data pipelines for training frontier AI models and agents.

Role type

Senior IC research engineer (RL evaluation & environments)

Builds

RL training data pipelines, evaluation systems, and executable agent environments

Domain

Frontier AI / Reinforcement Learning / Agent Systems

Deliverable

production ML models

Required skills

Python, data pipeline engineering, RL workflows, model evaluation design, failure mode analysis, reward hacking detection, agent environment construction

Preferred skills

RL post-training data, synthetic data generation, statistical testing, contamination detection, Verifiers framework experience

Technologies

Python, Verifiers, RL frameworks

Responsibilities

Design challenging evaluations for coding and long-horizon agents; Build executable environments with realistic task setup; Develop scoring logic and verifiers to detect reward hacks; Trace model failures to distinguish limitations from environment issues; Build and scale RL data pipelines for task generation; Collaborate with researchers to validate training data improvements

Seniority

Senior, hands-on IC