Senior Software Engineer, ML Infrastructure
Core
Designing, building, and operating production-grade inference infrastructure for a full-stack generative AI platform serving enterprise and government organizations.
Role type
Senior IC ML Infrastructure Engineer
Builds
SambaNova Suite (on-premises and cloud inference platform)
Domain
Generative AI / Large Language Model Inference
Deliverable
production ML models
Required skills
distributed systems design, LLM inference techniques, Python, systems design, concurrency, algorithms, data structures
Preferred skills
experience with vLLM, TensorRT-LLM, or SGLang
Technologies
RDU architecture, vLLM, SambaStack, SambaCloud
Responsibilities
Design and productionize advanced inference techniques (speculative decoding, prompt caching, long-context inference); Integrate and adapt serving frameworks to RDU architecture; Own the public inference API surface; Build accuracy verification and regression infrastructure; Partner with ML, compiler, and product teams to ship features; Contribute to technical design and architectural decisions
Seniority
Senior, hands-on IC
