Quality Analyst
Core
Build and maintain answer keys, audit moderator decisions, calibrate vendor QA teams, and analyze quality data to improve content safety on Roblox.
Role type
Senior IC Quality Analyst (Content Safety)
Builds
Labeled image cases, calibration materials, and quality reports for the Trust & Safety platform
Domain
Internet platform / Content moderation / Trust & Safety
Deliverable
production ML models
Required skills
Content policy judgment, Answer key construction, Vendor QA calibration, Root-cause analysis, Statistical sampling, Data analysis with spreadsheets, Cross-functional collaboration
Preferred skills
Experience with outsourced teams, Writing executive briefs
Technologies
Spreadsheets
Responsibilities
Build and maintain the answer key for moderator training; Make final calls on disputed decisions; Calibrate vendor QA teams; Find root causes and trends in quality data; Recommend fixes via stakeholder briefs; Report on quality with accuracy metrics; Use outside quality signals like user appeals; Confirm that fixes worked via re-audit
