Software Engineer, Evals
Core
Building large-scale evaluation pipelines, observability infrastructure, and quality measurement systems for AI assistants and agents.
Role type
Senior backend and infrastructure engineer (AI evals & observability)
Builds
Evaluation pipelines, trace enrichment systems, durable telemetry pipelines, and dashboards for AI quality.
Domain
Enterprise AI / Workforce AI / Distributed Systems
Deliverable
production ML models | infrastructure
Required skills
Go, Python, Java, C++, distributed systems, data pipelines, production services, observability systems, cloud-native infrastructure, backend architecture, reliability engineering, cost optimization, security.
Preferred skills
LLM applications, evals, tracing, data warehouses, workflow orchestration, ML infrastructure.
Responsibilities
Design and build large-scale evaluation pipelines; Evaluate frontier model releases and OSS model drops; Build agent observability infrastructure; Own backend systems from design to production rollout; Partner with product, ML, and infrastructure engineers; Improve the quality loop by connecting eval results to product improvements; Mentor other engineers.
Seniority
Senior, hands-on IC
