Member of Engineering (Pre-training / Data Research)
Core
Improving the quality of pretraining datasets (natural language and source code) for training LLMs and coding agents via synthetic data generation and data mix optimization.
Role type
Senior IC machine-learning engineer (data research & engineering)
Builds
Large-scale, high-quality pretraining datasets for model training
Domain
Artificial Intelligence / Large Language Models / Data Engineering
Deliverable
production ML models
Required skills
Large Language Models (LLM) architecture understanding, transformer architectures, data ablations, scaling laws, mid-training and post-training techniques, training reasoning and agentic models, evals tracking model capabilities, trillion-scale pretraining dataset construction, data curation, deduplication, data mixing, tokenization, curriculum design, impact of data repetition, Python programming, prompt engineering, large-scale GPU cluster management, distributed data pipelines
Preferred skills
Author of scientific papers on applied deep learning, LLMs, or source code generation
Technologies
Python, Large GPU clusters, distributed data pipelines
Responsibilities
Follow latest research on LLMs and data quality, design and implement complex data generation pipelines, collaborate with Pretraining, Posttraining, Evals, and Product teams, conduct and analyze data ablations or training experiments
Seniority
Senior, hands-on IC