Research, Pre-Training Data
Core
Designing and implementing methods for sourcing, curating, and analyzing large-scale pre-training datasets to underpin next-generation AI models.
Role type
Pre-training data researcher (research/engineering hybrid)
Builds
Pre-training datasets and data systems for collaborative general intelligence models
Domain
Artificial Intelligence / Machine Learning / Data Engineering
Deliverable
production ML models
Required skills
Python, deep learning frameworks (PyTorch, TensorFlow, JAX), distributed training, data curation, data quality metrics, statistical analysis
Preferred skills
Probability and statistics, large-scale text/code/multimodal dataset analysis, data ethics and safety frameworks, open source contributions, PhD in CS/ML/Physics/Mathematics
Technologies
PyTorch, TensorFlow, JAX
Responsibilities
Design techniques for curating and filtering large-scale text, code, and multimodal data; Develop data quality metrics to measure coverage and diversity; Collaborate with teams to scale data processing systems; Investigate and mitigate data risks (privacy, safety, licensing); Evaluate dataset improvements via downstream model effects; Publish research and share code/datasets
Seniority
Mid-to-Senior level (Bachelor's minimum, PhD preferred)
