Research, Pre-Training Data
Core
Design and implement methods for sourcing, curating, and analyzing large-scale pre-training datasets (text, code, multimodal) to underpin frontier AI models.
Role type
Pre-training data researcher (IC)
Builds
Pre-training datasets and data systems for frontier AI models
Domain
Artificial Intelligence / Machine Learning
Deliverable
production ML models
Required skills
Python, deep learning frameworks (PyTorch, TensorFlow, JAX), distributed training, data curation, data quality metrics, data ethics and safety
Preferred skills
Probability and statistics, large-scale web data processing, dataset construction, open-source contributions, PhD or equivalent industry research experience
Technologies
PyTorch, TensorFlow, JAX
Responsibilities
Design techniques for curating and filtering large-scale data; Develop data quality metrics to measure coverage and diversity; Collaborate with infrastructure teams to scale data processing; Investigate and mitigate data risks (privacy, safety, licensing); Evaluate dataset improvements via downstream model analysis; Publish research and share code/datasets
Seniority
Senior, hands-on IC