Senior Software Engineer - AI Inference Performance
Core
Senior engineer optimizing LLM/VLM inference performance on NVIDIA GPU-accelerated systems to improve latency, throughput, and efficiency.
Role type
Senior IC machine-learning systems engineer (inference performance)
Builds
High-performance inference runtimes, CUDA kernels, and distributed serving software for NVIDIA platforms
Domain
AI/ML inference, GPU computing, distributed systems
Deliverable
production ML models
Required skills
CUDA programming, Python, Rust, C++, GPU architecture (Tensor Cores, memory hierarchy), profiling (Nsight Systems/Compute), speed-of-light/roofline modeling, distributed systems, model parallelism, quantization, speculative decoding
Preferred skills
Open-source contributions (TensorRT-LLM, vLLM, SGLang), AI-agent-supported performance workflows, published research
Technologies
CUDA, CUTLASS, Triton, PyTorch, NCCL, TensorRT-LLM, vLLM, SGLang
Responsibilities
Lead end-to-end analysis of LLM/VLM inference processes; Build speed-of-light and roofline models; Profile workloads to eliminate bottlenecks; Tune serving hyperparameters and techniques; Build and optimize performance-critical kernels; Establish repeatable benchmarks and manage performance regression gates
Seniority
Senior, hands-on IC
