Distributed Training and Inference Engineer
Core
Build, optimize, and maintain the critical software stack for large-scale AI training and serving workloads, from low-level CUDA/ROCm runtimes to high-level frameworks like JAX and PyTorch.
Role type
Senior IC distributed training and inference engineer
Builds
Production ML training and inference systems for next-generation LLMs and generative AI models
Domain
AI infrastructure, distributed systems, GPU computing
Deliverable
production ML models
Required skills
Python, C++, distributed training systems, CUDA, ROCm, JAX, PyTorch, profiling tools, multi-node GPU cluster management, memory efficiency optimization, kernel-level debugging
Preferred skills
XLA/JAX stack internals, distributed serving frameworks, GPU kernel optimization, accelerator-aware model partitioning, low-level C++ building blocks
Technologies
CUDA, ROCm, JAX, PyTorch, vLLM, TensorRT, FasterTransformer, NCCL, XLA, Nsight, DTensor, GSPMD
Responsibilities
Maintain and optimize critical ML libraries and frameworks across multiple environments; Build and improve the entire ML software stack from drivers to tooling; Ensure efficient sharding and partitioning for large-scale distributed training; Integrate and validate modules for runtime correctness and scalability; Profile compilation graphs and workloads to eliminate bottlenecks; Troubleshoot complex hardware–software interaction issues
Seniority
Senior, hands-on IC