Data Engineer
Core
Designing and developing robust, scalable data pipelines and products for a security-critical domain.
Role type
Data Engineer
Builds
Production data pipelines and data products
Domain
Data engineering in a security-critical domain
Deliverable
production ML models
Required skills
Python, PySpark, Airflow, GitHub, Databricks, ETL pipeline development, data quality assurance
Preferred skills
Delta tables, Parquet files, machine learning, LLMs
Responsibilities
Design and develop data pipelines and data products; Build, optimize, and maintain ETL flows for production; Work with storage and management of large datasets in modern data architectures; Ensure data quality, traceability, and reliability; Collaborate with other engineers, analysts, and business representatives; Contribute to continuous improvement of platforms, workflows, and solutions
