Applied Science Manager, Artificial General Intelligence , Quality Automation
Core
Lead a team of Applied Scientists to develop quality strategies, auditing frameworks, and automated evaluation systems for Large Language Models (LLMs) and multimodal systems.
Role type
Senior Applied Science Manager (Quality & Evaluation)
Builds
Automated quality assessment systems (LLM-as-a-Judge), auditing frameworks, and high-quality training/evaluation data for Amazon Nova models.
Domain
Artificial General Intelligence (AGI), Large Language Models (LLMs), Quality Assurance
Deliverable
production ML models | product features
Required skills
Machine learning management, statistics, auditing methodologies, automated evaluation system design, LLM architecture design, rubric creation, team mentorship, root cause analysis
Preferred skills
Building machine learning models/algorithms, developing complex software systems involving deep learning and computer vision
Technologies
LLM-as-a-Judge, Large Language Models (LLMs), Deep Learning, Computer Vision
Responsibilities
Lead and mentor Applied Scientists in developing quality strategies and auditing frameworks; Design and maintain LLM-as-a-Judge systems for automated quality assessment; Oversee expert-level manual audits and meta-audits; Configure data collection workflows; Recruit, hire, and develop team members; Communicate quality feedback to stakeholders and senior leadership.
Seniority
Senior, hands-on IC with management responsibilities
