Engineering Manager, Evals
Core
Lead the team creating high-signal evaluation datasets for coding agents and building tools for engineers to write and run evaluations.
Role type
Engineering Manager, Evals
Builds
High-signal evaluation datasets (e.g., CursorBench), developer-friendly tools for running evals, and online evaluation systems.
Domain
AI/ML, Coding Agents, Software Engineering
Deliverable
production ML models
Required skills
People leadership and coaching, aligning research/product/data/infrastructure, defining quality signals and metrics, operating evaluation or measurement systems, data acumen, understanding model and agent behaviors.
Preferred skills
Experience shipping production systems, building evaluation systems (AI evals, experimentation platforms, ranking/relevance, search quality, reliability instrumentation).
Technologies
CursorBench, online evaluation systems
Responsibilities
Set the eval roadmap end-to-end, lead and grow a team of engineers and researchers, guide the evolution of evaluation benchmarks, define online quality signals and guardrails, integrate evals into launch and model training decisions.
Seniority
Manager, hands-on leadership
