CareerPlanSign in

Software Engineer L5 - AI Observability & Agent Evaluation

Los Gatos,California,United States of America💼 Full-time💰 $388,000–$388,000🗓 2026-09-16 → 2026-09-20

Core

Build frameworks, SDKs, and platforms for AI/ML observability, model evaluation, and vendor integration to make AI systems transparent and production-ready.

Role type

Senior IC platform engineer (AI observability & evaluation)

Builds

Reusable observability infrastructure, evaluation primitives, SDKs, connectors, and dashboards for ML/GenAI teams

Domain

AI/ML infrastructure, GenAI, LLM evaluation, distributed systems

Deliverable

production ML models | infrastructure

Required skills

Python, Java/Go/Scala, production observability, ML/LLM evaluation, distributed systems, cloud platforms, vendor integration

Preferred skills

ML/LLM observability tools, LLM/GenAI application development, agentic systems evaluation

Technologies

Prometheus, Grafana, Datadog, OpenTelemetry, ELK, OpenSearch, Jaeger, Tempo, AWS, GCP, Azure

Responsibilities

Build observability frameworks for metrics, logs, and traces across inference and batch; Create primitives for monitoring model performance, data quality, and drift; Develop evaluation frameworks for LLMs and agentic systems; Lead build-vs-buy evaluations and integrate vendor tooling; Build reusable libraries and templates for observability-by-default; Provide dashboarding, alerting, and SLO/SLI building blocks

Seniority

Senior, hands-on IC

Sourced via eightfold · Listed on CareerPlan, which tracks 70,000+ jobs from 20+ sources.