Applied Scientist, Artificial General Intelligence, AGI Data Services
Core
Developing quality strategies, auditing frameworks, and automated evaluation systems to ensure data integrity for Large Language Models (LLMs) and multimodal systems.
Role type
Applied Scientist (Quality Assurance & Evaluation)
Builds
Auditing frameworks, LLM-as-a-Judge systems, and automated quality assessment models
Domain
Artificial General Intelligence (AGI), Large Language Models (LLMs), Data Quality
Deliverable
production ML models
Required skills
Machine learning, statistics, quality assurance, auditing methodologies, automated evaluation systems, Java, C++, Python, SQL, RDBMS
Preferred skills
Algorithm implementation, top-tier conference publications
Technologies
LLM-as-a-Judge, Oracle, Data Warehouse
Responsibilities
Design auditing strategies with SOPs and quality metrics, perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, develop LLM-as-a-Judge architectures, configure data collection workflows, communicate quality feedback to stakeholders
Seniority
Mid-to-Senior, hands-on IC