Senior Data Engineer (Big Data Processing)
Core
Design and build scalable, robust data pipelines to ingest, parse, and process large datasets for national security threat identification and analysis.
Role type
Senior Data Engineer (Big Data Processing)
Builds
Mission-critical data analytics systems and high-performance data pipelines for national security use cases.
Domain
National Security / Defense / Big Data
Deliverable
production ML models | infrastructure
Required skills
Large-scale data pipeline development, Optimized ETL code, Scala, Java, Apache Spark, Apache Kafka, Hadoop, Elastic stack, Kudu, Modern data architecture design, Data modeling
Preferred skills
Technical leadership, Mentoring junior engineers, Stakeholder communication
Technologies
Spark, Kafka, Hadoop, Elastic stack, Kudu
Responsibilities
Design and build scalable data pipelines, Optimize performance of large-scale pipelines for low-latency and high-throughput, Ensure data integrity and governance, Troubleshoot complex issues in critical pipelines, Implement best practices in testing and orchestration, Lead technical discussions to shape data products, Mentor junior data engineers
Seniority
Senior, hands-on IC with mentorship responsibilities
