Senior Deep Learning Software Engineer, Inference
Core
Design, build, and optimize GPU-accelerated software for high-performance deep learning frameworks (SGLang, vLLM) to serve large-scale language and generative AI models.
Role type
Senior Deep Learning Software Engineer (Inference)
Builds
NVIDIA inference libraries (vLLM, SGLang, FlashInfer) and model serving pipelines for LLMs and Generative AI.
Domain
Artificial Intelligence / High-Performance Computing / GPU Accelerators
Deliverable
production ML models
Required skills
C/C++ programming, software design, deep learning inference optimization, multi-GPU communications, CUDA programming, performance profiling and debugging, CPU/GPU architecture knowledge
Preferred skills
Python, training DL models, performance modeling, building products for enterprise customers, open-source contributions (PyTorch)
Technologies
CUDA, NCCL, NVSHMEM, OAI Triton, CUTLASS, SGLang, vLLM, FlashInfer, PyTorch
Responsibilities
Performance optimization and tuning of DL models across LLM, Multimodal, and Generative AI domains; Scaling DL model performance across NVIDIA accelerator architectures; Contributing features and code to NVIDIA inference libraries; Collaborating with cross-functional teams on inference optimization solutions
Seniority
Senior, hands-on IC
