Member of Technical Staff - Pre-Training Infra
Core
Build and scale distributed training systems to power frontier model pre-training for open foundation models.
Role type
Senior IC distributed training infrastructure engineer
Builds
Large-scale training runs and production-ready training systems for foundation models
Domain
AI/ML infrastructure, distributed systems, pre-training
Deliverable
production ML models
Required skills
distributed training systems, Megatron, DeepSpeed, large-scale model parallelism, GPU communication optimization, NCCL, debugging distributed stacks, training pipeline development
Preferred skills
experience with foundation model pre-training datasets, rapid experimentation workflows
Technologies
Megatron, DeepSpeed, NCCL
Responsibilities
Design and operate large-scale training runs, optimize training throughput and GPU utilization, debug performance bottlenecks in distributed stacks, build training pipelines for datasets and checkpointing, collaborate with researchers to productionize experimental ideas
Seniority
Senior, hands-on IC
