Senior ML Data Processing Developer
Core
Design and engineer end-to-end data pipelines that transform raw web-scale data into high-signal datasets for training the Scientist AI.
Role type
Senior ML Data Processing Developer
Builds
High-throughput data processing pipelines, quality scoring mechanics, and data curation tooling for AI training
Domain
AI/ML, Data Engineering, NLP
Deliverable
production ML models
Required skills
Python, distributed processing frameworks (Spark, Ray, Flink), pipeline orchestration (Airflow, Prefect, Dagster), data privacy implementation, content-safety filtering, evaluation-contamination prevention, large-scale unstructured text dataset handling
Preferred skills
ML model training/fine-tuning for data-quality tasks, LLM inference optimization, containerized deployment, ML experiment tracking, data licensing workflows
Responsibilities
Partner with Research to define and scale data pipelines; Build and maintain pipelines including deduplication, PII scrubbing, and toxicity removal; Develop scoring toolchains with heuristics and LLM-as-a-judge evaluators; Ensure compliance with Data Governance policies; Instrument pipelines with quality monitoring and alerting; Acquire large-scale text corpora and conduct coverage analyses; Design leakage detection mechanisms; Build internal tooling for dataset exploration