Principal Research Scientist - Evaluations
Core
Define and own the evaluation strategy for Canva's generative models across design, image, video, audio, and agentic workflows to ensure metrics reflect real user experience and product impact.
Role type
Principal Research Scientist (Evaluation Strategy & Measurement)
Builds
Shared evaluation layer, auto-raters, MLLM judges, and offline/online monitoring suites for generative quality.
Domain
Generative AI, Multimodal Models, Human-in-the-loop Evaluation
Deliverable
production ML models | research
Required skills
Evaluation framework design, Reward modelling, Preference learning, Multimodal generative model architecture, Human preference scaling, Technical direction across distributed teams
Preferred skills
Human perception/psychophysics, Bias and safety evaluation, Visual arts background
Technologies
Diffusion models, Transformers, VLMs, MLLMs, RLHF, RLAIF
Responsibilities
Establish evaluation gates for launch decisions, diagnose anomalous evaluation results, partner with Design Generation and Foundation Models teams, turn subjective creative judgement into measurable criteria, mentor senior research scientists, represent evaluation vision to leadership
Seniority
Principal, strategy & mentorship
