Staff Software Engineer (AI Evals)
Core
Evaluate the quality of interactions with modern coding agents (e.g., OpenAI Codex, Claude Code) to assess engineering judgment and reasoning rather than syntax.
Role type
Staff-level AI evaluation engineer
Builds
Evaluation frameworks and feedback loops for AI coding agents
Domain
AI safety and evaluation / Software Engineering
Deliverable
production ML models
Required skills
Staff/Principal-level engineering experience, Go, Python, Rust, TypeScript/JavaScript, experience with OpenAI Codex, Claude Code, Cursor, ability to evaluate code without full execution, high bar for engineering quality
Preferred skills
Experience with Cursor or similar AI-first IDEs, prompt design or evaluation workflows, mentoring senior engineers, defining engineering standards
Technologies
OpenAI Codex, Claude Code, Cursor
Responsibilities
Evaluate AI-generated coding interactions end-to-end, judge output usefulness and correctness, assess quality of explanations and reasoning, distinguish response quality levels, provide opinionated feedback on what worked or didn't, help define standards for great interactions
Seniority
Staff / Principal-level