Engineering Manager - (ML) - Evaluation Platform
Core
Lead the end-to-end evaluation lifecycle for Canva's AI features, including test-like evals, comparative evaluation services, production monitoring, and human labelling tooling.
Role type
Engineering Manager (ML Evaluation Platform)
Builds
A self-serve evaluation platform integrating test-like evals, Arena comparative evaluation, monitoring evals, and labelling tooling for AI features.
Domain
Generative AI, Machine Learning Evaluation, Platform Engineering
Deliverable
production ML models | product features
Required skills
Technical people leadership, ML evaluation expertise, LLM systems knowledge, distributed systems design, stakeholder management, first principles thinking, strategic visioning, hiring and resourcing
Preferred skills
Experience coaching senior/staff-level ICs, experience with GenAI evaluation, ability to navigate competing priorities
Technologies
ML evaluation frameworks, LLM systems, distributed systems
Responsibilities
Own and evolve the evaluation platform strategy, partner with AI feature teams to identify evaluation needs, drive delivery of large cross-team initiatives, coach a team of 9+ engineers, define team development methodology and rituals, plan goals and success metrics, champion knowledge sharing, run hiring and resourcing
Seniority
Senior, hands-on IC with people leadership