Data Engineer
Core
Build and extend an established Azure Databricks data platform, translating business requirements into finished ETL flows and data models.
Role type
Senior Data Engineer (Azure Databricks)
Builds
Production ETL pipelines, gold warehouse data models, and data products for business units
Domain
Cloud data engineering (Azure Databricks, PySpark, SQL)
Deliverable
production ML models | product features
Required skills
PySpark development, Spark SQL, data modeling (Kimball principles), CI/CD pipeline management (YAML), unit and integration testing, data documentation, data governance and access control, incident management
Preferred skills
Azure DevOps, ServiceNow
Technologies
Azure Databricks, Azure DevOps, PySpark, Spark SQL, YAML, ServiceNow
Responsibilities
Develop and maintain ETL pipelines in Databricks using PySpark and SQL; Collaborate with business stakeholders to gather requirements and deliver data products; Model data in the gold warehouse and document solutions; Configure and maintain CI/CD pipelines in Azure DevOps; Ensure data cleansing routines and access control; Monitor job alerts and handle incidents in ServiceNow to improve platform performance