Senior Technical Program Manager, Human-in-the-Loop Operations
Core
Own the end-to-end lifecycle of Roblox's AI data labeling and model evaluation ecosystem, scaling operations from manual workflows to AI-assisted models.
Role type
Senior Technical Program Manager (Human-in-the-Loop Operations)
Builds
Data infrastructure for AI model quality across creator tools, content discovery, in-experience AI, and 3D generative content
Domain
AI/ML operations, 3D gaming, generative content
Deliverable
production ML models
Required skills
Technical Program Management, data labeling/annotation strategy, distributed workforce management, ML lifecycle knowledge, SQL, Python, cross-functional alignment, critical thinking, AI tool proficiency
Preferred skills
LLM-based labeling pipelines, LLM-as-a-Judge evaluation, vendor platform evaluation, multimodal data evaluation, consumer platform scale experience, gaming platform knowledge
Technologies
SQL, Python, Cursor, Claude, Gemini, Label Studio, Scale AI, Snorkel AI
Responsibilities
Lead end-to-end data programs from requirements to dataset delivery; Define and maintain ground truth via labeling guidelines and quality frameworks; Manage multi-vendor distributed contractor workforces; Drive shift to AI-assisted workflows with Engineering; Partner cross-functionally to translate model needs into data programs; Own operational excellence and KPIs; Shape platform and tooling strategy
Seniority
Senior, hands-on IC
