Senior Software Engineer – AI Middleware
Core
Design, develop, and upstream-enable AI communication middleware and collective communication libraries (e.g., NCCL/RCCL) for distributed AI workloads over high-performance interconnects.
Role type
Senior Software Engineer (AI Middleware & HPC Systems)
Builds
AI communication middleware, collective communication libraries, and performance optimization for distributed training clusters.
Domain
AI/ML Infrastructure, High-Performance Computing (HPC), Networking
Deliverable
production ML models
Required skills
C/C++ systems programming, GPU communication stacks (CUDA/ROCm), NCCL/RCCL, distributed training optimization, profiling and tracing, collective communication concepts, open-source contributions
Preferred skills
PyTorch Distributed, DeepSpeed, Megatron-LM, libfabric/OFI, UCX, RDMA, RoCEv2, Ultra Ethernet, cluster-scale test infrastructure
Technologies
C, C++, CUDA, ROCm, NCCL, RCCL, PyTorch, TensorFlow, XLA, JAX, DeepSpeed, Megatron-LM, libfabric, UCX, RDMA, RoCEv2, Ultra Ethernet
Responsibilities
Design and implement performance-critical features for CCL enablement; Optimize distributed training performance across multi-node, multi-GPU configurations; Improve GPU communication paths including GPU-direct transfers and CPU/GPU synchronization; Profile distributed AI workloads to identify bottlenecks; Tune AI frameworks such as PyTorch Distributed, TensorFlow/XLA, JAX, DeepSpeed, and Megatron-LM; Develop benchmarks and microbenchmarks aligned with real model performance; Contribute upstream to AI communication and distributed training projects; Prototype and validate Ultra Ethernet capabilities for AI collective communication; Collaborate with kernel/driver, switch, performance, and systems teams; Support advanced escalations by analyzing traces and providing robust fixes.
Seniority
Senior, hands-on IC