Senior Data Engineer (Databricks on Azure)
Core
Design, build, modernise, and operate scalable data pipelines and document-processing capabilities for Tax technology platforms on Azure Databricks.
Role type
Senior hands-on IC data engineer (cloud data platforms)
Builds
Scalable ingestion, ETL/ELT, and document-processing pipelines for tax technology systems
Domain
Cloud data engineering (Azure Databricks) + Tax technology
Required skills
Python, Apache Spark/PySpark, Azure Databricks (Delta Lake, Unity Catalog, Workflows, ADLS Gen2, Data Factory), SQL, data modelling, REST API integration, CI/CD, automated testing, data quality/observability
Preferred skills
Legacy pipeline modernisation, document extraction/classification, AI-assisted development tools, streaming data pipelines
Technologies
Azure Databricks, Python, PySpark, Delta Lake, Unity Catalog, Azure Data Factory, ADLS Gen2, Git, Azure DevOps
Responsibilities
Design and build scalable data pipelines; migrate legacy workloads to cloud-native platforms; implement data quality and monitoring controls; develop document extraction and automation pipelines; optimise workloads for performance and cost; collaborate with architects and subject matter experts.
Seniority
Senior, hands-on IC
