Senior System Software Engineer, Agentic Inference - Dynamo
Core
Develop open source software to serve inference of trained AI models running on GPUs, focusing on agentic inference workloads including long-horizon reasoning, tool calling, and stateful multi-turn execution.
Role type
Senior System Software Engineer (Agentic Inference)
Builds
Dynamo's distributed inference frontend and disaggregated serving capabilities for vLLM, SGLang, and TensorRT-LLM
Domain
Generative AI inference, GPU-accelerated deep learning, distributed systems
Deliverable
production ML models
Required skills
Rust, Python, software design, debugging, performance analysis, test design, LLM API semantics, structured outputs, tool calling, context management, multimodal inputs
Preferred skills
Open-source AI inference framework contributions, GPU memory optimization, KV/prefix cache optimization, high-performance networking, agentic workload scheduling, agent harness integration
Technologies
vLLM, SGLang, TensorRT-LLM, NIXL, Rust, Python
Responsibilities
Develop open source software for GPU-based AI model inference; Contribute to disaggregated serving for inference engines; Innovate in inference-state management for long-running agents; Build and evolve distributed inference frontend; Balance objectives to build robust, scalable, high-performance software components
Seniority
Senior, hands-on IC
