Data Engineer - Foundational
Core
Build high-performance data infrastructure and ETL pipelines to power foundational deep learning models for autonomous defense systems.
Role type
Senior IC data engineer (video/defense)
Builds
Terabyte-scale data pipelines, storage solutions, and data loaders for ML training clusters
Domain
Defense technology, autonomous systems, video analytics
Deliverable
production ML models
Required skills
Distributed systems, ETL/ELT pipeline design, video data processing, GPU optimization, storage architecture, data versioning, parallel computing
Preferred skills
Experience with unstructured data (video, images, point clouds), systems thinking
Technologies
WebDataset, TFRecords, Parquet, Apache Spark, Ray, Apache Beam, MinIO, S3, DVC, Apache Iceberg, Pachyderm, PyTorch, JAX
Responsibilities
Build ETL/ELT pipelines for EO and IR video data, develop algorithms for sensor synchronization and alignment, architect storage-to-GPU pipelines, write distributed data processing jobs, implement automated data quality checks and versioning, assess and implement advanced storage solutions
Seniority
Senior, hands-on IC
