Pre-training Research Engineer
Core
Implement, scale, and improve byte-native and multimodal foundation models through pre-training and architecture research.
Role type
Senior IC pre-training research engineer
Builds
High-quality base models for real-world use cases at scale
Domain
AI infrastructure, multimodal AI, large language models
Deliverable
production ML models
Required skills
large-scale pre-training, model architecture design, distributed training, deep learning fundamentals, research experimentation, robust software engineering, GPU training
Preferred skills
JAX ecosystem, multi-node pre-training systems (FSDP, ZeRO, Megatron), end-to-end training pipeline ownership
Technologies
JAX, Flax, XLA, FSDP, ZeRO, Megatron
Responsibilities
Train large byte-native and multimodal foundation models across massive, heterogeneous corpora; Implement and evaluate new model architectures, training objectives, and optimization methods; Develop stable pre-training recipes and run scaling experiments for novel architectures; Conduct ablations and analyze training dynamics, model behavior, and base-model quality; Work with data and distributed training engineers to improve training efficiency, reliability, and scalability
Seniority
Senior, hands-on IC