AI Software Evaluation Engineer, Siri AI Quality
Core
Build systems to evaluate Siri's AI experiences, analyze evaluation data, and automate scenario generation and triage to catch quality issues early.
Role type
Senior IC AI software evaluation engineer (LLM/agents)
Builds
Evaluation datasets, automated screening/triage/fix tooling, and coverage analysis reports for Siri
Domain
Consumer AI / Voice assistants / Mobile computing
Deliverable
production ML models
Required skills
LLM-based application development, agentic workflows, AI/LLM evaluation design, SQL, Python, automation, CI/CD
Preferred skills
Agent frameworks (MCP servers), voice assistant evaluation, data visualization (Tableau), data warehouses (Snowflake), root-cause analysis
Technologies
Python, SQL, JSON, MCP servers, Tableau, Snowflake
Responsibilities
Investigate test scenario regressions and recurring issues; analyze gaps in app intents and features; design agentic workflows for scenario generation and auto-fix; build and report on dataset quality and issue trends
Seniority
Senior, hands-on IC
