Machine Learning Engineer, Core Evaluations
Core
Designing and developing comprehensive model evaluation pipelines, automated dashboards, and user studies for speech generation and recognition models.
Role type
Senior IC machine learning engineer (model evaluation)
Builds
Evaluation pipelines, automated dashboards, and metrics for ASR/TTS models
Domain
Social AI, speech technology
Deliverable
production ML models
Required skills
designing metrics for model performance, designing user studies on Mechanical Turk, model training and fine-tuning, statistical analysis, engineering and programming, ASR/TTS model experience, large-scale ML experience (>3B models with >1m hours of data)
Preferred skills
leading evaluation teams
Technologies
Mechanical Turk
Responsibilities
Designing model evaluation pipelines for development and production environments, designing user studies for subjective model evaluations, converting requirements into measurable metrics, designing and developing automated evaluation dashboards, training new models to capture evaluation metrics, communicating with model teams to improve designs based on results, communicating with data teams to decide on data types, communicating with product managers to ensure requirements are measured, growing the evaluation team
Seniority
Senior, founding member with leadership expectations