Senior Data Engineer
Core
Design and operate a modern, scalable, secure enterprise lakehouse architecture on AWS to enable advanced analytics, AI/ML initiatives, and operational reporting for DoD programs.
Role type
Senior hands-on IC data engineer (cloud data platform)
Builds
Enterprise lakehouse environments, data pipelines, event-driven ingestion solutions, and analytical dashboards
Domain
Federal government / Cloud data engineering
Deliverable
production ML models
Required skills
Apache Spark, Python, SQL, AWS services, Apache Airflow, Terraform, Kubernetes, data governance, data quality
Preferred skills
Apache Iceberg, Trino, dbt, Helm, GitOps, data visualization platforms, data catalog tools
Technologies
AWS (S3, Glue, EMR, Athena, SQS), Apache Spark, Apache Iceberg, Apache Airflow, Terraform, Kubernetes, Trino, Grafana
Responsibilities
Architect and manage cloud-based lakehouse environments; Develop and optimize Apache Spark data pipelines; Design event-driven data ingestion solutions; Optimize query performance across distributed engines; Orchestrate data workflows and ETL pipelines; Manage infrastructure deployment and automation; Develop and maintain data products and dashboards; Implement data quality, governance, and access controls; Provide technical leadership and mentorship to junior engineers
Seniority
Senior, hands-on IC with technical leadership