Automation Engineer
Core
Design, develop, and maintain data pipelines and orchestration workflows to ingest, process, and transform data for a national laboratory.
Role type
Senior IC data engineering automation engineer
Builds
Scalable data ingestion and processing pipelines integrating on-premises and cloud infrastructure
Domain
Government contracting, data engineering, infrastructure automation
Required skills
Python, SQL, REST API design, data orchestration (Airflow/Prefect), ETL/ELT (dbt/PySpark), Linux/Unix administration, security compliance
Preferred skills
Data virtualization (Denodo), Big Data (Hadoop/Spark/Kafka), Cloud platforms (AWS/Azure/GCP)
Technologies
Airflow, AWS, Ansible, Azure, Big Data, Cloud, ETL, Flow, GCP, Hadoop, Kafka, Linux, MongoDB, Oracle, Puppet, Python, PySpark, REST, SQL, Spark, Unix, dbt
Responsibilities
Evaluate and recommend data ingestion and pipeline tools; Design and implement pipelines for structured and unstructured data; Develop and manage orchestration workflows; Design, develop, and maintain APIs for integration; Support data architectures across on-premises and cloud platforms; Integrate pipelines with applications; Collaborate with stakeholders to deliver robust solutions; Implement and manage data virtualization solutions; Maintain data quality, security, and compliance; Document data engineering processes.
Seniority
Senior, hands-on IC