Data Engineer
Core
Design and maintain robust ETL/ELT pipelines, build reusable frameworks on Databricks, and automate cloud infrastructure to ingest structured and unstructured data for analytics.
Role type
Senior Data Engineer (Cloud & Databricks)
Builds
Scalable batch and real-time streaming data pipelines, Databricks frameworks, and AWS cloud infrastructure
Domain
Healthcare / Data Engineering
Deliverable
production ML models
Required skills
SQL, PySpark, Python, AWS (S3, Lambda, Glue, Redshift, Athena), Databricks (DLT, DABs), CloudFormation, Data Modeling (SCD), CI/CD
Preferred skills
Dimensional modeling, Data quality testing, Infrastructure automation
Technologies
Databricks, AWS, Python, PySpark, SQL, CloudFormation, GitHub Actions
Responsibilities
Design and maintain ETL/ELT pipelines for cloud lakehouses and data warehouses; Build reusable frameworks using Databricks Notebooks and Delta Live Tables; Automate infrastructure deployment using CloudFormation and GitHub Workflows; Optimize data layout and query execution across Redshift and Delta Lake; Establish automated data validation and schema enforcement; Partner with cross-functional teams in an Agile environment to support evolving data requirements.