AIML - Machine Learning Engineer, Evaluation - Responsible AI and Safety
Core
Senior Machine Learning Engineer responsible for evaluating safety, fairness, and inclusivity of Apple's generative AI features (LLMs, diffusion models, multimodal systems) and operationalizing risk assessments into measurable criteria.
Role type
Senior IC machine learning engineer (Responsible AI Evaluation)
Builds
Measurable safety criteria, evaluation datasets, data pipelines, and insights informing ship/no-ship product decisions
Domain
Generative AI, Large Language Models, Multimodal Systems, Responsible AI
Deliverable
production ML models
Required skills
Metrics development for safety/fairness, dataset design (human/synthetic), sampling strategies, human-in-the-loop methods, data pipeline implementation, automation of evaluation workflows, failure investigation, stakeholder communication
Preferred skills
Multimodal models, agentic systems, diverse languages/locales, AI compliance/regulatory experience (child safety, health, companionship), large-scale benchmarking
Technologies
LLMs, Diffusion models, Human-in-the-loop systems, Data pipelines
Responsibilities
Develop metrics for safety and fairness risks; Design datasets and implement data coverage solutions; Develop sampling strategies combining human annotation and auto-grading; Design and optimize data pipelines for end-to-end safety evaluations; Automate manual workflows to improve evaluation velocity; Analyze evaluation results and conduct failure investigations
Seniority
Senior, hands-on IC
