Member of Technical Staff - Data Ingestion Engineer
Core
Build and operate large-scale data ingestion systems to acquire, extract, normalize, and deliver web and large-scale data sources into reliable corpora for training frontier AI models.
Role type
Senior IC data ingestion engineer (distributed systems)
Builds
Large-scale data ingestion pipelines, specialized crawlers, and dataset delivery systems for pre-training
Domain
AI/ML pre-training, large-scale data acquisition, distributed systems
Deliverable
production ML models
Required skills
Web crawling, data ingestion, large-scale data acquisition, distributed systems design, experiment design, data analysis, system observability, code review, debugging
Preferred skills
LLM training intuition, tradeoff analysis, rapid iteration based on measurable impact
Technologies
Ray, Beam, Spark
Responsibilities
Build and operate ingestion systems for web crawling and extraction; Run experiments to evaluate crawling strategies and ingestion tradeoffs; Analyze ingested data to identify gaps and redundancy; Develop specialized crawlers for high-priority data sources; Review code and debug production issues
Seniority
Senior, hands-on IC
