Data Engineer - Managing Consultant
Core
Design, build, and optimize scalable, production-grade data ingestion, transformation, and analytics-ready pipelines using Databricks and AWS services for enterprise use cases.
Role type
Senior IC data engineer (cloud data platforms)
Builds
Production data pipelines, governed datasets, and semantic layers for enterprise reporting and advanced analytics
Domain
Public sector (IRS/Treasury) data engineering and cloud architecture
Deliverable
production ML models | product features | infrastructure
Required skills
Python, PySpark, SQL, Databricks (Delta Lake, Delta Live Tables), AWS (S3, Glue, Kafka/Kinesis), data governance, DevSecOps, CI/CD
Preferred skills
Databricks Unity Catalog, Delta Sharing, Informatica EDC/Axon, Terraform, GitHub Actions, Databricks Asset Bundles
Technologies
Databricks, AWS, Delta Lake, Kafka, Kinesis, EventBridge, Informatica EDC, Terraform, GitHub Actions
Responsibilities
Design and build scalable data ingestion and transformation pipelines; Engineer standardized batch and near real-time data pipelines; Execute full delivery lifecycle including intake, discovery, and technical design; Implement governed data pipelines with metadata, lineage, and data quality controls; Develop data engineering frameworks with observability and monitoring; Partner with platform teams to integrate pipelines with AWS services; Apply security and compliance controls aligned to IRS cybersecurity policies
Seniority
Senior, hands-on IC
