Lead AI Engineer 4C
Core
Design and execute comprehensive evaluation frameworks, benchmarks, and success metrics for large-scale AI models and agentic AI systems.
Role type
Senior AI Evaluation Engineer
Builds
Evaluation strategies, test datasets, and risk-mitigation plans for AI products
Domain
Artificial Intelligence / Machine Learning / Responsible AI
Deliverable
production ML models
Required skills
AI/ML evaluation, Human-in-the-loop evaluation design, Python, Data analysis, Experiment tracking, Prompt engineering, Model behavior analysis, Safety and risk assessment, Stakeholder communication, Agile delivery
Preferred skills
Advanced Analytics, AI/ML Ops, Prompt Design
Technologies
Python, Experiment-tracking frameworks
Responsibilities
Develop evaluation frameworks and benchmarks for AI models; Lead large-scale human evaluations and create test datasets; Analyze evaluation results to identify failure patterns and risks; Partner with researchers and product teams to align evaluation goals; Ensure evaluations meet safety standards and regulatory expectations.
Seniority
Senior, hands-on IC
