Technical Operations Analyst, Automation & Evaluations
Core
Build tools, datasets, dashboards, and automated workflows (using SQL, Python, and LLMs) to support data analysis, measurement, and operational improvements for the User Safety & Risk Operations team.
Role type
Senior IC Technical Operations Analyst (Automation & Analytics)
Builds
Datasets, dashboards, scripts, API integrations, and automated evaluation workflows for Trust & Safety operations.
Domain
AI Safety & Risk Operations
Required skills
SQL, Python, data analysis, dashboard creation, API integration, workflow automation, metric definition, evaluation design
Preferred skills
Prototype-to-production scaling, handling messy operational data, defining assessment criteria, resolving ambiguous cases
Technologies
SQL, Python, LLMs
Responsibilities
Define automation opportunities with partner teams; investigate operational problems using SQL/Python; build and maintain datasets and dashboards; define metrics to assess work quality and impact; build scripts and API integrations to automate tasks; evaluate automated workflows against expert-reviewed examples; hand off workflows with clear ownership and maintenance instructions.
Seniority
Senior, hands-on IC