AIML - Sr Software Engineer - AI, Evaluation
Core
Own and scale the evaluation platform for Apple Intelligence, Siri, and generative features, ensuring quality and reliability for launch decisions.
Role type
Senior IC software engineer (AI evaluation platform)
Builds
LLM-as-judge autograders, validation tools, and evaluation systems
Domain
Consumer AI / Large Language Models
Deliverable
production ML models
Required skills
Python, system design, API design, system testing and validation, debugging, monitoring, AI coding agents
Preferred skills
agentic systems, LLM-based applications, LLM-as-judge systems, offline evaluations, technical integrations, product planning
Technologies
Python, LLMs
Responsibilities
Own a core evaluation system end-to-end for multiple features; Design testing and validation strategies; Build and maintain integrations with annotation, compute, reporting, and benchmarking platforms; Partner with teams building LLM judges; Set the bar for using AI coding agents without compromising quality
Seniority
Senior, hands-on IC
