Senior/Staff Software Engineer - LLM Inference Systems & Open-Source
Core
Build and advance software ecosystem for large-scale LLM inference on Qualcomm platforms, focusing on open-source LLM serving systems and AI accelerator integration.
Role type
Senior/Staff Software Engineer (LLM Inference Systems & Open-Source)
Builds
Open-source LLM inference engines (e.g., llm-d), distributed serving frameworks, and Qualcomm AI accelerator integrations.
Domain
AI/ML Infrastructure, Distributed Systems, Hardware Acceleration
Deliverable
production ML models
Required skills
C++, Python, Go, Rust, Distributed systems, LLM inference systems, AI accelerator software, Cloud-native/Kubernetes infrastructure, ML compiler/runtime systems
Preferred skills
LLM inference engines, High-performance inference runtimes, Distributed LLM inference architectures, llm-d/KServe/Kubernetes operators, PyTorch internals/compilers, Hardware accelerator software stacks
Technologies
llm-d, KServe, Kubernetes, PyTorch, Triton, MLIR, RDMA, RoCE
Responsibilities
Contribute features and fixes to open-source LLM inference projects; Develop support for Qualcomm AI accelerators in serving frameworks; Design distributed LLM inference capabilities (prefill/decode disaggregation, KV-cache management, model parallelism); Integrate inference software with Kubernetes and cloud-native infrastructure; Profile and optimize end-to-end inference performance; Diagnose complex issues across the stack.
Seniority
Senior/Staff, hands-on IC
