Senior Data Engineer
Core
Design and deliver scalable, metadata-driven data solutions and orchestration processes within a modern Azure-based data ecosystem.
Role type
Senior IC data engineer (Azure/Databricks)
Builds
Metadata ingestion frameworks, orchestration processes, and reusable data services
Domain
Insurance / Data Engineering
Deliverable
production ML models
Required skills
Databricks (SQL & PySpark), Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure Functions, Airflow, dbt, Spark Declarative Pipelines, CI/CD for data, metadata-driven pipeline design, data governance
Preferred skills
Insurance data experience
Technologies
Databricks, Azure, Airflow, dbt, Spark
Responsibilities
Design and implement scalable metadata-driven services to optimise data pipelines; Build and maintain robust data solutions using Databricks and Azure services; Apply and embed best practices for data engineering, CI/CD pipelines, and DevOps; Collaborate with stakeholders to translate data needs into engineering solutions; Optimise existing data pipelines for performance and scalability; Provide technical guidance and mentoring to junior engineers; Act as escalation point for complex data issues
Seniority
Senior, hands-on IC with mentorship