Data Engineer
Core
Designing and building secure, high-quality data pipelines and ETL processes to power health research using UK health datasets.
Role type
Senior Data Engineer (Healthcare)
Builds
Secure, scalable data pipelines and research-ready data assets on Azure
Domain
Healthcare / Data Engineering
Deliverable
production ML models
Required skills
Python, PySpark, Azure, SQL, Data Modeling, Code Review, Data Governance, Workflow Orchestration (Dagster), Kubernetes, Databricks
Preferred skills
Healthcare coding systems (SNOMED CT, ICD, READ), Metadata Management, Security Principles
Technologies
Azure, PySpark, Python, GitHub, Dagster, Kubernetes, Databricks, SQL
Responsibilities
Building and maintaining data pipelines across Azure-based platform; Developing data transformation and ETL processes; Working with healthcare and NHS datasets to curate trusted data assets; Reviewing and approving code through pull requests; Producing documentation, source-to-target mappings, and supporting technical standards; Collaborating with researchers and stakeholders to understand requirements.
Seniority
Senior, hands-on IC