Member of Technical Staff - ML Research Engineer, Pre-Training Data
Core
Design and build data pipelines, mixtures, and synthetic data systems to fuel pre-training foundation models.
Role type
Senior IC ML Research Engineer (Pre-Training Data)
Builds
Pre-training data pipelines, synthetic data generation systems, and data mixtures/curricula
Domain
AI/ML, Pre-training data engineering
Deliverable
production ML models
Required skills
Pre-training data sourcing and curation, Synthetic data generation, Data mixture and curriculum design, Deduplication and decontamination, Experiment design and ablation, Production data pipeline development, ML fundamentals (PyTorch)
Preferred skills
Distillation strategies, Evaluation framework design, Post-training data work, On-device deployment, Startup experience
Technologies
Python, PyTorch
Responsibilities
Source and curate web-scale datasets, Build pipelines for filtering and decontamination, Design data mixtures for pre-training, Generate synthetic data using LLMs, Run proxy experiments to measure data impact, Build trusted evaluation sets
Seniority
Senior, hands-on IC
