CareerPlan

AI Evaluation Systems Engineering Lead

Mountain ViewFull-time$143k–$275k2026-11-17 → 2026-11-19

Core

Build evaluation infrastructure to help researchers improve frontier models and make reliable release decisions.

Role type

Senior IC engineering lead (distributed systems & ML evaluation)

Builds

End-to-end evaluation systems for benchmark configuration, model inference, grading, aggregation, and results publication

Domain

AI/ML research infrastructure, distributed systems, LLM evaluation

Deliverable

production ML models

Required skills

Distributed systems engineering, Python, C/C++/Java/JavaScript/C#, GPU-backed inference, distributed training, large-scale compute workloads, system architecture, team leadership, operational excellence, reproducibility validation

Preferred skills

LLM evaluation, benchmarking, model-based grading, reinforcement learning infrastructure, Ray/Kubernetes/Slurm, experimental reproducibility, hiring/mentoring engineers

Technologies

Ray, Kubernetes, Slurm, Python, C, C++, C#, Java, JavaScript

Responsibilities

Set technical direction and roadmap for evaluation systems; Build infrastructure for benchmarking and model evaluation; Accelerate research iteration and GPU efficiency; Ensure reproducible and trustworthy results; Maintain operational excellence and reliability; Contribute code and debug complex distributed problems; Build and mentor engineering teams; Align with research and engineering partners

Seniority

Senior, hands-on IC with leadership responsibilities