Research Engineer, Infrastructure, Numerics
Core
Design and build core systems enabling efficient large-scale model training with a focus on numerics, improving numerical foundations of distributed training stacks.
Role type
Senior IC infrastructure research engineer (numerics & distributed systems)
Builds
Distributed training infrastructure for large-scale LLMs, low-precision numerics implementations, and communication primitives
Domain
AI Infrastructure / Large Language Models / Distributed Systems
Deliverable
production ML models
Required skills
Deep learning frameworks (PyTorch, JAX), distributed systems, low-precision arithmetic (BF16, MXFP8, NVFP4), kernel optimization, mixed-precision arithmetic, scaling strategies (data/tensor/pipeline parallelism), internal orchestration systems
Preferred skills
Distributed frameworks (PyTorch/XLA, DeepSpeed, Megatron-LM), FP8/INT8/MX format implementation, open-source contributions, numerical optimization publications, large-scale model training experience
Technologies
PyTorch, JAX, BF16, MXFP8, NVFP4, DeepSpeed, Megatron-LM, XLA
Responsibilities
Design and optimize distributed training infrastructure for large-scale LLMs; Implement and evaluate low-precision numerics; Develop kernels and communication primitives for mixed/low-precision arithmetic; Collaborate with research teams on model architectures and training recipes; Prototype and benchmark scaling strategies; Contribute to internal orchestration and monitoring systems; Publish learnings via documentation or open-source
Seniority
Senior, hands-on IC