Data Engineer
Core
Design, build, and maintain scalable data pipelines and architectures for mission-critical programs within the Intelligence Community.
Role type
Data Engineer
Builds
Scalable ETL/ELT pipelines and cloud-based data architectures
Domain
Intelligence Community / Cloud Data Engineering
Deliverable
production ML models | infrastructure
Required skills
AWS data services (S3, Glue, Lambda, Redshift, EMR, Kinesis), SQL, Python, ETL/ELT pipeline development, big data technologies, data optimization, data governance
Preferred skills
Batch and real-time processing, indexing and partitioning strategies, API integration, Agile development
Technologies
AWS (S3, Glue, Lambda, Redshift, EMR, Kinesis), SQL, Python
Responsibilities
Design and develop scalable ETL/ELT pipelines; Build and optimize data workflows for batch and real-time processing; Ensure high availability and reliability of data pipelines; Optimize data storage and retrieval for performance and cost; Implement data validation, monitoring, and error-handling mechanisms; Integrate data solutions with enterprise applications and downstream analytics platforms
Seniority
Mid-to-Senior, hands-on IC
