Data Scientist
Core
Design evaluation harnesses, build production prediction models, and analyze LLM/agentic systems for an AIOps platform.
Role type
Senior IC data scientist (LLM evaluation & predictive modeling)
Builds
AIOps platform with locally-hosted language models, evaluation infrastructure, and production forecasting systems
Domain
IT Operations (AIOps) + Large Language Models (LLM)
Deliverable
production ML models | product features
Required skills
LLM evaluation design, predictive modeling, time-series forecasting, anomaly detection, Python, SQL, statistical experimentation, agentic system analysis, retrieval evaluation
Preferred skills
Small/local model optimization, red-teaming, IT operations domain knowledge, big-data stores
Technologies
Python, SQL, LLM evaluation frameworks, time-series libraries
Responsibilities
Design and own evaluation harnesses for LLM and agentic outputs; Build production prediction and trend capability from operational telemetry; Red-team the system for prompt injection and edge-case discovery; Monitor output-distribution and quality drift in production; Apply AIOps/NOC analysis for log anomaly detection and root-cause analysis; Quantify system economics and connect to customer-facing value metrics
Seniority
Senior, hands-on IC
