Principal Software Engineer – Large-Scale LLM Memory and Storage Systems
Core
Define the vision and roadmap for memory management of large-scale LLM and storage systems, designing a unified memory layer spanning GPU, host, SSD, and remote storage to support efficient LLM inference.
Role type
Principal Systems Engineer (LLM Infrastructure)
Builds
Unified memory layers and deep integrations with LLM serving engines (vLLM, SGLang, TensorRT-LLM) for KV-cache offload, reuse, and remote sharing.
Domain
AI Infrastructure / Distributed Systems / High-Performance Storage
Deliverable
production ML models
Required skills
Large-scale distributed systems, memory hierarchies (GPU HBM, DRAM, SSD), distributed caching/KV systems, networked I/O (RDMA, NVMe-oF, NVLink), system profiling and optimization, C/C++, Python
Preferred skills
Open-source contributions to LLM serving systems, unified memory/storage layer design, publications or patents in LLM systems/memory-disaggregated architectures
Technologies
Rust, Python, vLLM, SGLang, TensorRT-LLM, RDMA, NVLink, GPU HBM, SSD
Responsibilities
Design and evolve a unified memory layer spanning multiple tiers; Architect and implement integrations with LLM serving engines; Co-design interfaces for disaggregated prefill and KV-cache sharing; Partner with GPU/networking teams to exploit low-latency technologies; Mentor engineers and set technical direction.
Seniority
Principal, hands-on IC with strategy & mentorship
