Software Engineer, Agentic Runtime
Core
Design and operate core runtime services for multi-turn orchestration, tool calling, model routing, memory, streaming, and safety to power Glean's AI agents and assistant experiences at scale.
Role type
Senior IC software engineer (distributed systems & agentic runtime)
Builds
Low-latency, reliable, and secure runtime services for AI agents and assistants
Domain
Enterprise AI / Agentic AI / Distributed Systems
Deliverable
production ML models
Required skills
Python, Go, Java, Javascript, Typescript, C++, Kubernetes, cloud platforms (GCP, AWS, Azure), event/streaming systems (Pub/Sub, Kafka), caching (Redis), LLM/agents building blocks (tool/function calling, structured outputs, streaming, model selection/routing), observability (tracing, metrics, logs, OpenTelemetry)
Preferred skills
policy/guardrails, multi-tenant isolation, rate-limiting, concurrency control, cost optimization
Technologies
gRPC, WebSockets, OpenAI, Anthropic, Google Gemini, OpenTelemetry
Responsibilities
Own impactful runtime problems end-to-end from architecture to production launch; Build and evolve core services for session lifecycle, streaming responses, structured tool execution, memory/state, and policy/guardrails; Design for performance, correctness, and cost by reducing latency and optimizing token/tool budgets; Integrate with leading LLM providers and internal evaluation frameworks; Harden the platform with fault isolation, retries, timeouts, circuit-breaking, and graceful degradation; Instrument deep observability and create playbooks/SLOs for high availability; Collaborate with product, quality, and application teams to prioritize roadmap investments
Seniority
Senior, hands-on IC
