Pre-Training Infrastructure
Core
Building and optimizing the software stack for massive GPU clusters, high-throughput storage systems, and distributed training infrastructure to power frontier-scale AI models.
Role type
Senior IC infrastructure engineer (distributed training & HPC)
Builds
Software stack for massive GPU clusters, high-throughput storage systems, and distributed training parallelism
Domain
AI infrastructure / High-Performance Computing / Distributed Systems
Deliverable
infrastructure
Required skills
C++, Python, CUDA, NCCL, distributed computing, GPU programming, performance profiling, benchmarking, InfiniBand, NVLink, storage systems
Preferred skills
PyTorch, large-scale ML infrastructure, networking, architectural decision making
Technologies
Python, C++, CUDA, NCCL, PyTorch, InfiniBand, NVLink
Responsibilities
Design, implement, test, and optimize distributed training infrastructure for large-scale GPU clusters; Profile, benchmark, and debug performance bottlenecks across compute, memory, networking, and storage subsystems; Optimize collective communication libraries for emerging NVLink and InfiniBand topologies; Collaborate with hardware teams to optimize for next-generation accelerators; Gather data and insights to develop the pretraining compute roadmap.
Seniority
Senior, hands-on IC