Machine Learning Evaluation Engineer
Core
Design and deploy evaluation strategies, benchmarking systems, and failure analysis workflows for multi-modal foundation models and computer vision algorithms to ensure high-quality customer experiences.
Role type
Senior IC machine learning evaluation engineer
Builds
Comprehensive evaluation pipelines, automated failure analysis workflows, and curated evaluation datasets
Domain
Consumer electronics, computer vision, multi-modal AI
Deliverable
production ML models
Required skills
Machine Learning evaluation, Computer Vision, Generative AI, Python, Statistical analysis, Data curation, LLM-as-Judge, Failure analysis, Metric design, Pipeline orchestration
Preferred skills
Vision-Language Models, Prompt tuning, Human-in-the-loop methodologies
Technologies
Python, LLMs, VLMs, Cloud platforms
Responsibilities
Design and build comprehensive evaluation pipelines for large datasets; Develop and calibrate subjective evaluations using LLM-as-Judge and human grading; Identify root causes in failure cases and develop failure taxonomies; Design cloud-based workflows and dashboards for failure analysis; Refine strategies for curating high-quality datasets and ground truth; Assess data quality, diversity, and coverage gaps; Partner with model training teams to provide actionable insights.
Seniority
Senior, hands-on IC
