Data Engineer for Databricks Platform (all humans)
Core
Build and optimize data solutions on the Databricks platform, manage centralized data ingestion frameworks, and ensure data quality for the bank's digital future.
Role type
Data Engineer (Databricks Platform)
Builds
Centralized data ingestion frameworks, end-to-end data pipelines, and data quality monitoring solutions.
Domain
Financial Services / Data Engineering
Deliverable
production ML models | product features | dashboards & analysis
Required skills
PySpark, SQL, Delta Lake, data ingestion patterns, ETL/ELT processes, data modeling, Git, CI/CD pipelines
Preferred skills
HashiCorp Terraform, DataOps/MLOps practices, Unity Catalog, workflow orchestration tools
Technologies
Databricks, PySpark, Delta Lake, Spark SQL, structured streaming, AWS, MS Azure
Responsibilities
Work hands-on with the Databricks platform to build and optimize data solutions; Manage, operate, and continuously improve a centralized data ingestion framework; Design, build, and maintain data ingestion pipelines end-to-end; Troubleshoot and resolve data quality incidents; Develop and implement data quality frameworks and monitoring solutions; Expand platform capabilities by evaluating and adopting new Databricks features; Contribute to infrastructure-as-code practices and CI/CD pipeline improvements.
Seniority
Mid-level (2-5 years experience)
