Data Engineer
Core
Designing, developing, and managing large-scale data workflows and RAG (Retrieval-Augmented Generation) pipelines for US federal government clients.
Role type
Senior Data Engineer (Generative AI & Data Infrastructure)
Builds
Scalable RAG pipelines, ETL/ELT workflows, and data systems in AWS.
Domain
US Federal Government / Generative AI / Cloud Data Engineering
Deliverable
production ML models | infrastructure
Required skills
Python, ETL/ELT, AWS services (S3, Lambda, Glue, EMR, ECS/EKS), ElasticSearch, Prefect, data modeling, data governance
Preferred skills
Apache NiFi, vector databases (OpenSearch, Pinecone, Weaviate), containerized workflows (Docker, Kubernetes), LLM/generative AI production systems, distributed systems
Technologies
AWS, ElasticSearch, Prefect, NiFi, Python, Pandas, PySpark, Docker, Kubernetes, OpenSearch, Pinecone, Weaviate
Responsibilities
Design and maintain RAG pipelines including document ingestion, indexing, and retrieval flows; Build and optimize ETL/ELT pipelines in AWS; Develop and orchestrate workflows using Prefect; Implement indexing and search patterns using ElasticSearch; Ensure data quality, lineage, and security; Monitor system performance and troubleshoot distributed data systems
Seniority
Senior, hands-on IC (via careerplan.io/jobs/4706625006-data-engineer-at-accenturefederalservices)
