Senior Software Engineer, LLM Inference
Core
Optimizing LLM inference performance and implementing algorithms for hybrid Mamba-Transformer and mixture-of-experts models across NVIDIA hardware.
Role type
Senior IC software engineer (LLM inference optimization)
Builds
Production-grade inference software for data center and edge devices
Domain
AI/Deep Learning, GPU Systems, Distributed Computing
Deliverable
production ML models
Required skills
CUDA programming, Triton, GPU kernel tuning, distributed inference (expert parallelism, communication-compute overlap), deep learning architectures (Transformers, SSMs, MoE), performance profiling (Nsight, roofline models), open-source library development (vLLM, SGLang, Dynamo, FlashInfer)
Preferred skills
LLM quantization (FP8, NVFP4, MXFP8), multi-token predictors, sparse hybrid models, performance modeling and simulation-to-silicon correlation
Technologies
CUDA, Triton, PyTorch, Nsight, vLLM, SGLang, Dynamo, FlashInfer
Responsibilities
Implement and optimize inference algorithms for LLM and omnimodal architectures; Profile inference pipelines and correlate simulation predictions with real hardware; Write and tune GPU kernels for operators like fused MoE layers and quantized GEMMs; Solve distributed inference problems including expert parallelism and multi-node deployment; Build production-grade software inside major open-source libraries; Own optimization features end-to-end from scoping through delivery
Seniority
Senior, hands-on IC
