Data Engineer
Core
Develop, deploy, and maintain secure, scalable data pipelines and platform services within the AWS and Databricks ecosystem for a financial regulatory agency.
Role type
Data Engineer (AWS/Databricks)
Builds
Enterprise data pipelines, SQL-based reporting solutions, and platform services
Domain
Financial regulation, Cloud Infrastructure
Deliverable
production ML models | infrastructure
Required skills
Python, SQL, PySpark, AWS services (S3, IAM, CloudWatch, Glue, Athena, Redshift), ETL/ELT design, CI/CD (GitHub Actions, Terraform, Databricks Asset Bundles), Unity Catalog, data governance
Preferred skills
AWS or Databricks certifications
Technologies
AWS, Databricks, Apache Spark, PySpark, GitHub Actions, Terraform, Unity Catalog
Responsibilities
Support ingestion, transformation, and publication of enterprise datasets; Develop and maintain SQL-based reporting and analytics solutions; Implement and maintain CI/CD pipelines; Support cloud operations including infrastructure automation, logging, and monitoring; Participate in platform incident response and troubleshooting
Seniority
Junior to Mid-level, hands-on IC