Data Scientist– AI Infra & Evaluation Foundations
Core
Design metrics, pipelines, and methodology to evaluate and trust agentic AI systems, bridging AI research and production infrastructure.
Role type
Senior Data Scientist (AI Infra & Evaluation)
Builds
Centralized evaluation frameworks, agent lifecycle tools, and production-grade monitoring pipelines for agentic AI features.
Domain
Agentic AI systems, AI infrastructure, and evaluation methodologies.
Deliverable
production ML models
Required skills
Agentic systems architecture, evaluation methodologies, production-grade coding, trace-first diagnostics, product intuition, stakeholder partnership
Preferred skills
Centralized platform/infra experience, microservice architectures, CI/CD pipelines, agent observability tooling, TypeScript
Technologies
LangGraph, LangChain, SoTA SDKs, LangSmith, Langfuse, TypeScript
Responsibilities
Design evaluation metrics and pipelines for AI agents, standardize agent lifecycle and regression testing, partner with teams to translate expectations into datasets, prototype end-to-end eval pipelines, anticipate future failure modes in agent architectures
Seniority
Senior, hands-on IC