Systems Engineer - Evaluation Engineering
Core
Design and build the execution engine and infrastructure for a massive-scale distributed platform that orchestrates multi-model judging pipelines and agentic simulations.
Role type
Senior distributed systems engineer (platform)
Builds
High-throughput agentic simulation orchestration, multi-model judging pipelines, and real-time observability dashboards
Domain
AI/ML infrastructure, distributed systems, cloud-native architecture
Deliverable
production ML models
Required skills
Python (asyncio), Java, gRPC/Protobuf, GraphQL, REST, Kubernetes, AWS/GCP/Azure, PostgreSQL, Kafka, Terraform, CI/CD
Preferred skills
Relational data modeling, event-driven architectures, Agentic RAG platforms, containerized deployment pipelines
Technologies
Python, Java, gRPC, Protobuf, GraphQL, REST, FastAPI, Kubernetes, Istio, Linkerd, AWS, GCP, Azure, PostgreSQL, Kafka, AWS SQS/SNS, RabbitMQ, Redis Streams, Terraform, GitHub, ArgoCD
Responsibilities
Architect and scale the core asynchronous engine for orchestrating parallel agent simulations; Design and build self-service infrastructure, CLI tools, and internal APIs; Design, build, and maintain highly performant, type-safe APIs; Build robust data pipelines to ingest and transform high-volume execution traces; Own deployment topologies using declarative infrastructure and continuous delivery; Implement deep observability and smart scheduling layers to prevent cascading failures
