Data Engineer - Databricks
Core
Design, develop, and optimise scalable data pipelines and transformations within a modern Databricks lakehouse architecture to support data modernisation in financial services.
Role type
Senior Data Engineer (Databricks)
Builds
Scalable data pipelines, Bronze/Silver/Gold data models, and cloud-based lakehouse architectures
Domain
Financial Services / Insurance
Deliverable
production ML models
Required skills
Databricks, PySpark, SQL, dbt, Delta Lake, ETL/ELT pipeline development, cloud platforms (Azure/AWS), CI/CD, data testing
Preferred skills
Insurance or financial services domain experience, AI-assisted development tools (GitHub Copilot, Databricks Genie)
Technologies
Databricks, Python, PySpark, SQL, dbt, Delta Lake, Azure, AWS, GitHub Copilot, Databricks Genie
Responsibilities
Design and optimise data pipelines, build and maintain data transformations, develop scalable data models, support data migration from legacy platforms, implement data quality checks and performance optimisation
Seniority
Senior, hands-on IC