Data Engineer (Databricks)
Core
Design, develop, and maintain scalable data pipelines and ETL/ELT processes using Databricks and cloud technologies to help clients make data-driven decisions.
Role type
Data Engineer (Databricks)
Builds
Modern data solutions, scalable data pipelines, and ETL/ELT processes for clients.
Domain
Cloud data platforms, Microsoft Azure ecosystem, Lakehouse architectures.
Required skills
Python, PySpark, SQL, Databricks, ETL/ELT pipeline design, Delta Lake, Lakehouse architectures, data modeling, cloud data platforms (Azure), CI/CD, Git, DevOps.
Preferred skills
Unity Catalog, data governance, streaming technologies, real-time data processing, consulting experience.
Technologies
Databricks, Python, PySpark, SQL, Delta Lake, Azure Data Factory, Azure Data Lake Storage, Synapse Analytics, Microsoft Fabric, Git.
Responsibilities
Design and develop scalable data pipelines; Build modern data solutions using Databricks; Develop and optimize data transformations; Work with Delta Lake and lakehouse architectures; Implement data ingestion, transformation, integration, and quality processes; Collaborate with Data Architects, Data Scientists, BI specialists, and business stakeholders; Apply best practices around data quality, performance, governance, and security.
Seniority
Mid-Senior, hands-on IC