Staff Data Scientist
Core
Own the definition, execution, and evolution of evaluation frameworks, metrics, and analytical methodologies to assess AI/ML feature performance in real-world deployments, ensuring measurement rigor and decision support.
Role type
Staff Data Scientist (Evaluation & Analytics)
Builds
Evaluation frameworks, analytics platforms, KPI audits, and self-serve reporting tools
Domain
Transportation safety, Computer Vision, Edge Computing, AI/ML
Deliverable
production ML models | dashboards & analysis | research
Required skills
Probability and statistics, Python (OOP, algorithms), SQL (complex queries, indexing), Error analysis, Bias analysis, Experiment design (A/B tests), Statistical rigor, Data visualization, AI tool evaluation
Preferred skills
Cloud platforms (AWS Kinesis, EKS), Python web frameworks (Flask, Django), Large-scale noisy real-world datasets
Responsibilities
Design and maintain offline/online evaluation frameworks for AI/ML features; Define and validate KPIs and audit methodologies; Perform deep error and bias analysis to identify failure modes; Conduct large-scale analytical studies on feature performance and data quality; Design and review experiments including controlled rollouts and A/B tests; Build and maintain tools, dashboards, and automation frameworks for scaling audits; Mentor junior data scientists on statistical rigor and experiment design
Seniority
Staff, hands-on IC with mentorship
