Researcher - Pretraining Data Team
Core
Design and curate high-quality datasets to power frontier AI models, developing novel data collection strategies and improving dataset quality and integrity.
Role type
Senior IC researcher (pretraining data)
Builds
High-quality datasets for training and evaluation, scalable data pipelines, and lightweight tools for dataset auditing and versioning.
Domain
Artificial Intelligence / Large Language Models / Data Engineering
Deliverable
production ML models
Required skills
Python, Pandas, NumPy, Spark, Ray, Apache Beam, statistics, exploratory data analysis, data pipeline development, dataset curation, data ablation experiments
Preferred skills
Experience with unstructured or semi-structured large-scale datasets, data analysis, data engineering
Technologies
Python, Pandas, NumPy, Spark, Ray, Apache Beam
Responsibilities
Design and curate high-quality datasets; Develop novel data collection strategies; Create high-quality datasets for training and evaluation; Run experiments on new datasets (data ablations); Develop and maintain scalable data pipelines; Analyze real-world datasets to assess quality and diversity; Build lightweight tools for dataset auditing and visualization
Seniority
Senior, hands-on IC
