Data Ops Engineer
Core
Design, build, and maintain real-time data ingestion pipelines ensuring reliable flow, data quality, observability, and scalability for mission-critical objectives.
Role type
Senior Data Operations Engineer (Data Engineering)
Builds
Real-time data ingestion pipelines and monitoring software for streaming data from diverse sources.
Domain
Defense and Intelligence (DoD/IC) data operations
Deliverable
production ML models | infrastructure
Required skills
Python, Bash, data pipeline design, backpressure management, error handling, root cause analysis, incident management, data encryption, authentication, authorization, data classification
Preferred skills
Experience with SIGINT, GEOINT, or multi-INT data sources
Technologies
NiFi, Kafka, Grafana, Prometheus, Apache Flink, Spark Streaming, Snowflake, Elasticsearch, MQTT, JMS, Windows, Linux (RedHat)
Responsibilities
Design and maintain real-time data ingestion pipelines; build resilient pipelines with backpressure and retry strategies; monitor data quality and pipeline health; automate monitoring and alerting for data issues; perform root cause analysis for data outages; partner with security teams to enforce encryption and access controls.
Seniority
Senior, hands-on IC