Data Platform Engineer - Data Operations
Core
Design, build, and maintain Python-based ETL pipelines and data infrastructure for ingesting, processing, and curating multi-sensor data including flight data, rosbags, video, and external deliveries.
Role type
Senior Data Platform Engineer (Data Operations)
Builds
ETL pipelines, data catalogs, metadata databases, backend APIs, dashboards, and self-service data access tools.
Domain
Aviation data, robotics, computer vision, multi-sensor data processing
Deliverable
production ML models | infrastructure
Required skills
Python (typing, testing), ETL pipeline design, SQL, schema design, PostgreSQL, object storage (GCS/S3), Docker, Unix/Linux shell scripting, CI/CD pipelines
Preferred skills
Synthetic data generation, GenAI workflows, data versioning, computer-vision annotation formats, advanced GCP services, robotics data formats, large-scale multimodal data
Responsibilities
Design and maintain typed, tested Python software and ETL pipelines; Build tools for video subsampling, frame extraction, dataset packaging, and self-service data access; Design and maintain metadata databases and data catalogs; Own technical workflows for data curation and labeling including vendor coordination; Develop backend APIs, dashboards, and data-access tools; Establish software engineering practices including testing, typing, documentation, logging, and CI/CD pipelines; Migrate legacy data, manage access, and automate recurring support tasks
