AI 品質評価エンジニア・AI 製品開発
Core
First dedicated QA engineer for AI products, focusing on establishing quality mechanisms for LLM-based products with uncertainty and non-determinism.
Role type
Senior IC AI Product Quality Assurance Engineer
Builds
Production AI/LLM products for Toyota's mobility ecosystem
Domain
Automotive / AI / LLM
Deliverable
production ML models
Required skills
LLM QA and testing, test design, evaluation criteria definition for ambiguous outputs, test process automation, cross-functional collaboration with MLOps/data engineers, business-level English and Japanese
Preferred skills
Scenario testing, red teaming, user feedback analysis, natural language processing fundamentals
Technologies
LLM, MLOps
Responsibilities
Define quality dimensions (accuracy, consistency, safety, fairness, UX), execute scenario and exploratory testing, analyze LLM output trends and failure patterns, design quality evaluation using user feedback, standardize and automate evaluation processes, collect and aggregate evaluation data, build and operate feedback loops with development teams, document quality risks
Seniority
Senior, hands-on IC with functional leadership