Senior Data Engineer – Azure & Databricks
Core
Design, build, and optimize scalable data pipelines and modern data solutions using Azure Databricks, PySpark, and Delta Lake within a lakehouse architecture.
Role type
Senior hands-on IC Data Engineer
Builds
Scalable data pipelines, Delta tables, medallion architectures, and optimized data models
Domain
Cloud data engineering (Azure ecosystem)
Required skills
Azure Databricks, PySpark, Delta Lake, Medallion Architecture, Azure SQL, Advanced SQL/T-SQL, SQL performance tuning, Azure Data Factory, CI/CD, Version control, Dimensional Data Modelling, Parquet, Lakehouse architecture
Preferred skills
Unity Catalog, Databricks data governance, Kafka, Event-driven data ingestion, Data quality, Data lineage, Access control, Operational monitoring, Azure DevOps, Infrastructure as Code, Platform automation, Azure cost optimization, FinOps practices
Technologies
Azure Databricks, PySpark, Delta Lake, Azure SQL, Azure Data Factory, Parquet, Unity Catalog, Kafka, Azure DevOps
Responsibilities
Design and develop data solutions using Azure Databricks and PySpark; Build and maintain Delta tables and medallion architectures; Optimize SQL performance including execution plans and indexing; Contribute to CI/CD and automated deployment; Act as a technical reference for other Data Engineers
Seniority
Senior, hands-on IC with mentoring responsibilities