Senior Network Production Engineer, AI Supercomputing
Core
Own the production health, operations, and reliability of multi-rail Ethernet backend networks (MRC, RDMA) for frontier-scale AI supercomputers training advanced models.
Role type
Senior Network Production Engineer (AI Supercomputing)
Builds
Production-quality software, automation, and monitoring systems for network validation, diagnosis, and fleet-wide change management.
Domain
AI Supercomputing / High-Performance Computing / Datacenter Networking
Deliverable
production ML models
Required skills
Large-scale datacenter/HPCC network operations, Ethernet/RDMA/RoCEv2, Linux systems, Python/Go/C++, incident response, telemetry analysis, automation, root cause analysis
Preferred skills
GPU training clusters, MRC multi-rail Ethernet, NCCL/CUDA/GPUDirect RDMA, ConnectX NICs, SONiC/SAI, InfiniBand, Kubernetes/Slurm, streaming telemetry (gNMI/Prometheus/Datadog)
Technologies
MRC, RDMA, RoCEv2, Ethernet, Linux, Python, Go, C++, Kubernetes, Slurm, Prometheus, Datadog, gNMI, eBPF, ConnectX, SONiC, SAI, InfiniBand, NCCL, CUDA
Responsibilities
Operate MRC Ethernet backend network at frontier scale; lead high-severity incident response; build health models correlating switch/NIC/host telemetry; develop automated mitigation mechanisms; improve large-job reliability and placement policies; automate fleet operations and compliance checks; drive cross-company technical investigations.
Seniority
Senior, hands-on IC