Senior Deep Learning Software Engineer, Inference
Core
Design, build, and optimize GPU-accelerated software for high-performance deep learning inference frameworks (SGLang, vLLM) to serve large-scale LLM and Generative AI models.
Role type
Senior IC deep learning software engineer (inference)
Builds
High-performance deep learning inference frameworks and libraries (SGLang, vLLM, FlashInfer)
Domain
Artificial Intelligence / High-Performance Computing / GPU Acceleration
Deliverable
production ML models
Required skills
C/C++ programming, software design, GPU programming (CUDA), performance optimization, multi-GPU communications (NCCL), deep learning frameworks (PyTorch, vLLM, SGLang), model serving pipelines
Preferred skills
Python, performance modeling and profiling, architectural knowledge of CPU and GPU, experience with training and deploying DL models in production, experience building products for enterprise customers
Technologies
CUDA, OAI Triton, CUTLASS, NCCL, NVSHMEM, SGLang, vLLM, FlashInfer, PyTorch
Responsibilities
Performance optimization, analysis, and tuning of DL models in LLM, Multimodal, and Generative AI domains; Scale performance of DL models across different NVIDIA accelerator architectures; Contribute features and code to NVIDIA's inference libraries; Work with cross-collaborative teams on framework and library optimization
Seniority
Senior, hands-on IC
