Senior Network Production Engineer, AI Supercomputing
Core
Own the availability, performance, and operational readiness of the MRC Ethernet backend network for AI supercomputing training clusters.
Role type
Senior IC network production engineer (AI supercomputing)
Builds
Production network health models, automated mitigation mechanisms, and fleet-wide change management systems.
Domain
High-performance computing (HPC) and AI training infrastructure
Deliverable
production ML models
Required skills
Large-scale datacenter network operations, Ethernet/RDMA/RoCEv2, Linux systems, Python/Go/C++, incident leadership, distributed systems debugging
Preferred skills
GPU training cluster operations, MRC/Multi-rail Ethernet, NCCL/CUDA/GPUDirect RDMA, Kubernetes/Slurm, streaming telemetry (gNMI/Prometheus/Datadog)
Technologies
MRC, ConnectX-class NICs, SONiC, SAI, InfiniBand, Kubernetes, Slurm, Prometheus, Datadog, eBPF
Responsibilities
Operate on-call rotation for high-severity network incidents; Diagnose failures across GPUs, NICs, switches, and distributed workloads; Build automation for network validation and remediation; Investigate collective-performance degradation in training jobs; Establish network qualification gates for large training pools.
Seniority
Senior, hands-on IC
