Member of Technical Staff - Pre-Training
Core
Research and build solutions for large language model pre-training, focusing on algorithms, scaling laws, data processing, optimizers, and model architecture.
Role type
Senior IC machine-learning engineer (pre-training)
Builds
Open foundational models and training infrastructure
Domain
Artificial Intelligence / Large Language Models
Deliverable
production ML models
Required skills
large-scale ML, distributed training, scaling laws, data processing, deep learning frameworks, Python, software engineering for large-scale systems, ETL workflows
Preferred skills
low-level optimizations, high-level model design, navigating research vs engineering trade-offs
Technologies
PyTorch
Responsibilities
Design and run scientific experiments to advance understanding of scaling large language models and data efficiency; Implement state-of-the-art methods from the deep learning literature; Optimize the training infrastructure for efficient scaling; Lead small research projects independently while collaborating on larger initiatives; Contribute across the entire stack from low-level optimizations to high-level model design
Seniority
Senior, hands-on IC
