Software Engineer, Training Infrastructure
Core
Design, build, and operate distributed training infrastructure for frontier-model training, including scheduling, orchestration, checkpointing, and recovery.
Role type
Senior IC distributed systems engineer (ML training infrastructure)
Builds
Distributed training infrastructure for Tinker and internal frontier-model training
Domain
Artificial Intelligence / Machine Learning Infrastructure
Deliverable
production ML models
Required skills
Python, C++/Rust/Go, distributed systems, PyTorch, performance profiling, parallelism strategies, memory optimization, kernel tuning, communication-level tuning, failure detection, job recovery, multi-tenant isolation, observability, LoRA fine-tuning, reinforcement learning workloads
Preferred skills
large-scale LLM training, mixture-of-experts architectures, cluster orchestration (Kubernetes/Slurm), fault-tolerant checkpointing, profiling tooling, open-source ML systems contributions
Technologies
PyTorch, Kubernetes, Slurm, LoRA, vLLM, SGLang, Megatron
Responsibilities
Design and operate distributed training infrastructure; improve training throughput and cost efficiency; support new model architectures; build automated failure detection and observability; support multi-tenant LoRA fine-tuning and RL workloads; collaborate with API and product teams
Seniority
Senior, hands-on IC