Sr. Software Engineer, Pretraining Data
Core
Build and own the data systems behind frontier models' initial training, including large-scale crawling, data platform, and pipeline infrastructure.
Role type
Senior IC software engineer (distributed data systems)
Builds
Distributed data infrastructure, high-throughput pipelines, and crawling systems for model pretraining
Domain
AI/ML infrastructure, large-scale data processing
Deliverable
production ML models
Required skills
distributed systems, large-scale data infrastructure, data pipeline orchestration, web crawling, data quality algorithms, technical leadership, system architecture
Preferred skills
outlier depth in infrastructure, high-slope engineer, end-to-end ownership, complex system debugging, cross-team project leadership
Technologies
distributed systems, data platforms, crawling infrastructure
Responsibilities
Design and scale distributed infrastructure for training-ready datasets; build models to classify, rank, and clean training data; optimize large-scale web crawling infrastructure; lead complex engineering initiatives and mentor engineers
Seniority
Senior, hands-on IC with leadership