Pretraining Data
Core
Design, curate, and evaluate large-scale datasets to train frontier AI models and develop scalable data pipelines.
Role type
Senior IC data engineer (AI/ML)
Builds
High-quality training datasets, scalable data ingestion/processing pipelines, and lightweight auditing tools.
Domain
Artificial Intelligence / Deep Learning / Data Engineering
Deliverable
production ML models
Required skills
Python, Pandas, NumPy, Spark, Ray, Apache Beam, statistics, exploratory data analysis, data pipeline development, dataset curation, data quality assessment
Preferred skills
Advanced statistics, large-scale unstructured data processing, data visualization, technical communication
Technologies
Python, Pandas, NumPy, Spark, Ray, Apache Beam
Responsibilities
Create high-quality datasets for training and evaluation; run experiments on new datasets to assess impact; develop and maintain scalable data pipelines for ingestion, processing, filtering, and annotation; analyze real-world datasets to assess quality, diversity, and relevance; build lightweight tools for dataset auditing, visualization, and versioning; collaborate with Safety, Ethics, and Governance teams to ensure datasets meet standards for quality, privacy, and responsible AI practices.
Seniority
Senior, hands-on IC