Data Engineer, Data Quality & Provenance
Core
Design and operate scalable batch and streaming pipelines to transform petabyte-scale fleet and simulation data into trusted, reproducible datasets for autonomous driving model training and safety analysis.
Role type
Senior Data Engineer (Data Quality & Provenance)
Builds
Petabyte-scale data products, versioned datasets, metadata catalogs, and distributed compute infrastructure for autonomous vehicle development. (via careerplan.io/jobs/102a31c7-44a5-48bc-8c6e-1f4434e83974-data-engineer-data-quality-provenance-at-wayve)
Domain
Autonomous driving, AI platforms, sensor data, geospatial data
Deliverable
production ML models
Required skills
Python, SQL, distributed data systems, workflow orchestration (Spark, Flyte, Airflow), cloud data platforms, data quality & lineage, data governance
Preferred skills
Autonomous vehicles, ADAS, robotics, mapping, sensor-rich systems, geospatial data, multimodal datasets
Technologies
Spark, Flyte, Airflow, object storage, cloud infrastructure
Responsibilities
Design and operate scalable batch and streaming pipelines; Build data models, catalogues, indexes and query capabilities; Create versioned, reproducible datasets; Develop workflows for data ingestion, transformation, curation, and quality validation; Partner with engineering teams to define schemas and data contracts; Improve reliability and cost efficiency of storage and compute workloads.
