Senior Staff Machine Learning Engineer
Core
Define evaluation strategy and success metrics for GenAI systems, build data flywheels for continuous model improvement, and lead cross-functional quality initiatives for Airbnb Assistance Engineering.
Role type
Senior Staff Machine Learning Engineer (GenAI Evaluation & Data Flywheel)
Builds
Evaluation frameworks, data flywheels, and production pipelines for GenAI systems
Domain
Travel marketplace, Generative AI, Customer Assistance
Deliverable
production ML models
Required skills
GenAI/LLM system orchestration, evaluation methodology design, data pipeline construction, model lifecycle management, A/B testing, human-in-the-loop evaluation
Preferred skills
Customer support workflow optimization, scalable evaluation platform architecture, agile practice for applied AI
Technologies
LLM fine-tuning, RAG, automated testing frameworks, dataset versioning tools
Responsibilities
Define evaluation strategy and success metrics for GenAI systems, build and scale evaluation frameworks with bias/drift controls, design data flywheels for continuous improvement, lead cross-functional quality initiatives, develop production pipelines for dataset creation and model monitoring, drive technical decisions for evaluation infrastructure
Seniority
Senior Staff, hands-on IC with strategic leadership
