Data Engineer, Data Quality & Provenance
Core
Build the data foundation for autonomous-driving development by creating trusted, reproducible datasets from fleet and simulation data.
Role type
Senior Data Engineer (Data Quality & Provenance)
Builds
Scalable batch and streaming pipelines, data models, catalogs, and versioned datasets for ML and autonomy teams.
Domain
Autonomous driving / Embodied AI / Robotics
Deliverable
production ML models
Required skills
Python, SQL, distributed data systems, Spark, Flyte, Airflow, cloud object storage, data quality, lineage, observability
Preferred skills
autonomous vehicles, ADAS, robotics, geospatial data, time-synchronised sensor data, ML training workflows
Technologies
Spark, Flyte, Airflow
Responsibilities
Design and operate scalable batch and streaming pipelines for multimodal data; Create data models, catalogs, and query capabilities; Build versioned, reproducible datasets for training and evaluation; Develop workflows for data ingestion, transformation, and quality validation; Partner with engineers to define schemas and data contracts; Establish standards for lineage and cost management; Improve performance of large-scale storage and compute workloads.
Seniority
Senior, hands-on IC
