Software Engineer, Pretraining Data
Core
Build platforms and pipelines to convert raw web, code, and multimodal data into datasets for frontier pretraining.
Role type
Senior IC software engineer (pretraining data infrastructure)
Builds
Scalable data pipelines, web-crawling systems, and data quality models for frontier model training
Domain
AI/ML infrastructure, large-scale distributed systems, data engineering
Deliverable
production ML models
Required skills
distributed systems architecture, large-scale data infrastructure, end-to-end system design, complex infrastructure debugging, web-crawling system design, data quality modeling, pipeline orchestration, observability, reproducibility, lineage management
Preferred skills
intuition for large-scale distributed systems, understanding of pretraining data impact on model quality
Technologies
web-crawling systems, data pipelines, orchestration tools, observability systems
Responsibilities
Build and scale web-crawling systems for URL discovery, scheduling, fetching, parsing, and extraction; Develop high-throughput data-quality pipelines for classifying, ranking, filtering, and cleaning data; Design scaling-ladder experiments for data mixtures and quality depth; Own scalable pipelines and health systems for data lineage and freshness
Seniority
Senior, hands-on IC