CareerPlan

Senior Software Engineer – AI Middleware

Remote – US onlyRemoteFull-time2026-04-10 → 2026-11-16
Remote · from United States

Core

Design, develop, and upstream-enable AI communication middleware and collective communication libraries (e.g., NCCL/RCCL) for distributed AI workloads over high-performance interconnects.

Role type

Senior Software Engineer (AI Middleware & HPC Systems)

Builds

AI communication middleware, collective communication libraries, and performance optimization for distributed training clusters.

Domain

AI/ML Infrastructure, High-Performance Computing (HPC), Networking

Deliverable

production ML models

Required skills

C/C++ systems programming, GPU communication stacks (CUDA/ROCm), NCCL/RCCL, distributed training optimization, profiling and tracing, collective communication concepts, open-source contributions

Preferred skills

PyTorch Distributed, DeepSpeed, Megatron-LM, libfabric/OFI, UCX, RDMA, RoCEv2, Ultra Ethernet, cluster-scale test infrastructure

Technologies

C, C++, CUDA, ROCm, NCCL, RCCL, PyTorch, TensorFlow, XLA, JAX, DeepSpeed, Megatron-LM, libfabric, UCX, RDMA, RoCEv2, Ultra Ethernet

Responsibilities

Design and implement performance-critical features for CCL enablement; Optimize distributed training performance across multi-node, multi-GPU configurations; Improve GPU communication paths including GPU-direct transfers and CPU/GPU synchronization; Profile distributed AI workloads to identify bottlenecks; Tune AI frameworks such as PyTorch Distributed, TensorFlow/XLA, JAX, DeepSpeed, and Megatron-LM; Develop benchmarks and microbenchmarks aligned with real model performance; Contribute upstream to AI communication and distributed training projects; Prototype and validate Ultra Ethernet capabilities for AI collective communication; Collaborate with kernel/driver, switch, performance, and systems teams; Support advanced escalations by analyzing traces and providing robust fixes.

Seniority

Senior, hands-on IC