Senior Software Engineer, Agent Eval Platform
Core
Build the judgement layer of an agent evaluation platform that scores multi-step agent trajectories in stateful enterprise environments to enable self-improving AI agents.
Role type
Senior IC backend/infrastructure engineer (agent evaluation platform)
Builds
Eval orchestration runtime, agent observability/tracing system, and stateful simulation environments for enterprise systems.
Domain
AI Agents, Enterprise Automation, Observability, Distributed Systems
Deliverable
production ML models | product features | infrastructure
Required skills
Distributed systems design, Workflow orchestration, Observability internals, Concurrent/async programming, Data-intensive pipelines, gRPC/protobuf design
Preferred skills
Python, Go, OpenTelemetry, LLM evaluation methodology
Technologies
Python, Go, OpenTelemetry, gRPC, protobuf, Temporal, Airflow, Argo
Responsibilities
Build runtime for multi-turn agent scenario execution with high concurrency and isolation; Design span data models for agent trajectory observability; Develop stateful simulation environments for enterprise systems; Establish reliability floors and SLOs for the evaluation harness.
Seniority
Senior, hands-on IC