CareerPlan

Senior Data Engineer

Horsham, UKRemoteFull-time2026-09-15 → 2026-11-16
HybridSecurity clearance required

Core

Design and deliver scalable Lakehouse architectures and production-grade data pipelines for defence, government, and national security clients using Databricks and cloud-native technologies.

Role type

Senior hands-on IC data engineer with technical leadership

Builds

Scalable Lakehouse architectures, production data pipelines, and enterprise data platforms

Domain

Defence, government, and national security data engineering

Required skills

Databricks, PySpark, Apache Spark, Delta Lake, Lakehouse architecture (Medallion), ETL/ELT, Azure/AWS, data governance, Unity Catalog, data modelling, CI/CD

Preferred skills

Technical leadership, mentoring, stakeholder engagement, emerging technology adoption

Technologies

Databricks, Apache Spark, PySpark, Delta Lake, Unity Catalog, Azure, AWS

Responsibilities

Design and build scalable data pipelines using Databricks and Spark; Develop modern Lakehouse architectures with Medallion design principles; Build robust ETL/ELT processes for large datasets; Drive data quality, monitoring, and operational support; Optimise platform performance, scalability, and cost; Integrate data platforms with cloud services and enterprise systems; Implement data governance, security, and access controls; Lead technical delivery across data engineering workstreams; Collaborate with data scientists and stakeholders to deliver end-to-end solutions; Support DevOps and CI/CD practices.

Seniority

Senior, hands-on IC with leadership responsibilities