Applied Scientist, Foundational AI
Core
Develop quality strategies, auditing frameworks, and automated evaluation systems (LLM-as-a-Judge) to ensure data integrity for Amazon Nova Large Language Models and multimodal systems.
Role type
Applied Scientist (Foundational AI / Quality Assurance)
Builds
Automated quality assessment systems, auditing SOPs, and evaluation rubrics for LLMs
Domain
Artificial General Intelligence (AGI), Large Language Models (LLMs), Multimodal Systems
Deliverable
production ML models
Required skills
Machine learning, statistics, quality assurance, auditing methodologies, automated evaluation systems, Java, C++, Python, SQL, RDBMS
Preferred skills
Algorithm implementation, top-tier conference publications
Technologies
LLM-as-a-Judge, Amazon Nova models, Oracle, Data Warehouse
Responsibilities
Design auditing strategies with SOPs and quality metrics, perform expert-level manual audits and meta-audits, develop LLM-as-a-Judge architectures, configure data collection workflows, conduct root cause analysis on data quality issues, coach auditors to improve quality capabilities
Seniority
Mid-to-Senior, hands-on IC