Senior Software Engineer, DGX Cloud AI Infrastructure
Core
Lead the bring-up, triage, benchmarking, analysis, and optimization of distributed training and inference workloads for large-scale LLMs across NVIDIA GPU platforms.
Role type
Senior IC software engineer (distributed AI systems & GPU infrastructure)
Builds
Large-scale AI clusters, communication libraries, model frameworks, and inference/training stacks for LLM workloads
Domain
Generative AI, High-Performance Computing (HPC), GPU clusters, Distributed Systems
Deliverable
production ML models
Required skills
Expert Python and C/C++, NCCL, CUDA-aware distributed execution, multi-GPU/multi-node debugging, large-scale distributed system architecture, RDMA stack (IB verbs, UCX, libfabric), GPU cluster fabrics (NVLink, NVSwitch, PCIe, RoCE, InfiniBand), profiling tools (Nsight Systems, NCCL tests)
Preferred skills
Experience building acceptance tests, benchmark harnesses, regression gates, cluster qualification tooling, resilience/failure-attribution systems for datacenter-scale infrastructure
Technologies
PyTorch, NeMo, Megatron, TensorRT-LLM, Nsight Systems, NCCL tests, NVLink, NVSwitch, PCIe, RoCE, InfiniBand
Responsibilities
Lead bring-up, validation, and debugging of large-scale AI clusters; Tune and benchmark AI pre-training, post-training, and inference workloads; Profile and optimize end-to-end workload performance across compute, memory, networking, and communication layers; Analyze scaling efficiency for distributed LLM workloads; Own root-cause analysis of complex failures in large distributed environments; Define and build the resilience and failure-attribution stack; Build repeatable benchmark suites and qualification workflows; Mentor engineers and drive technical standards
Seniority
Senior, hands-on IC with technical leadership