Senior Staff Machine Learning Engineer, Data & Eval
Core
Define evaluation strategy and build data flywheels for GenAI systems powering customer support products.
Role type
Senior Staff Machine Learning Engineer (Data & Eval)
Builds
Evaluation frameworks, data pipelines, and quality systems for LLMs and GenAI agents.
Domain
Customer Support x Artificial Intelligence (CSxAI)
Deliverable
production ML models
Required skills
GenAI/LLM orchestration, evaluation methodology, data pipeline design, ML lifecycle management, A/B testing, human-in-the-loop evaluation
Preferred skills
Customer support system optimization, evaluation platforms at scale, agile practice for applied AI
Technologies
LLMs, RAG, synthetic data, automated regression testing, rubric-based grading
Responsibilities
Define evaluation strategy and success metrics for GenAI systems; Build and scale evaluation frameworks with controls for bias and drift; Design the data flywheel for continuous improvement; Lead cross-functional quality initiatives; Develop and productionize pipelines for dataset creation and model monitoring; Drive technical decisions and architecture for evaluation infrastructure
Seniority
Senior Staff, hands-on IC with strategic leadership
