Data Engineer (Databricks)
Core
Build and maintain the DeltaFlow data platform by extending data set coverage, designing new datasets, and implementing SQL transformations using Databricks technologies.
Role type
Data Engineer (Databricks)
Builds
DeltaFlow data platform datasets and transformations
Domain
Private banking / Data Engineering
Required skills
Unity Catalog, Delta Lake, Change Data Feed, PySpark, SQL, Azure DevOps CI/CD, Data Modeling (Dimensional)
Preferred skills
TOML configuration, Kafka/Spark pipelines, Data Quality frameworks
Responsibilities
Extend DeltaFlow data set coverage by integrating new sets using established configuration models; Design new datasets; Write dataset-specific TOML configurations; Implement SQL transformation files for each dataset; Validate new datasets against quality/quarantine models; Deploy changes via existing Azure DevOps CI/CD pipeline; Verify datasets against DQ categories (completeness, uniqueness, validity, referential integrity, cross-level consistency); Document integrated datasets according to existing inventory conventions.