Staff Applied Scientist - Dashboards
Core
Defining and guaranteeing the quality of Datadog's AI-native Dashboards product, focusing on evaluation strategies, measurement systems, and agent success metrics for observability data.
Role type
Staff Applied Scientist (AI Evaluation & Measurement)
Builds
AI-native control surface for metrics, logs, and traces; evaluation datasets, golden traces, and regression harnesses for dashboards and widgets.
Domain
Observability / AI / GenAI
Deliverable
production ML models
Required skills
ML system evaluation at scale, experimentation design, measurement system architecture, tool-selection accuracy optimization, retrieval relevance improvement, cross-functional technical leadership, product-driven research-to-production execution.
Preferred skills
Experience with non-deterministic agent trajectories, multi-turn interaction evaluation, context efficiency optimization.
Technologies
Dashboards, agents, visualization widgets, data sources.
Responsibilities
Own the evaluation strategy for Dashboards and sister teams; Build eval datasets, golden traces, and regression harnesses; Drive measurable improvements to retrieval relevance and tool-selection accuracy; Provide technical leadership through design reviews and mentorship.
Seniority
Staff, hands-on IC with strategic scope