Principal AI Evaluations Platform Engineer
Core
Build and operate the evaluation platform supporting large-scale AI model training and development for Microsoft AI.
Role type
Senior IC platform engineer (AI evaluation infrastructure)
Builds
Reliable evaluation infrastructure for large-scale model training
Domain
Artificial Intelligence / Distributed Systems
Deliverable
production ML models
Required skills
Python, C++, Go, Rust, distributed systems operations, debugging distributed failures, on-call rotation
Preferred skills
GPU workloads, distributed training, Ray, Kubernetes, Slurm, observability tooling (Datadog, Prometheus, Grafana), LLM evaluation
Technologies
Python, C++, Go, Rust, Ray, Kubernetes, Slurm, Datadog, Prometheus, Grafana
Responsibilities
Provide dedicated coverage for the evaluation stack during European hours; Develop benchmark configurations, problem sets, graders, and evaluation runners; Improve evaluation throughput and scheduling efficiency; Build monitoring, alerting, and dashboards; Partner with researchers to define requirements and coordinate launches; Contribute to technical design and code reviews.
Seniority
Senior, hands-on IC