Senior Data Engineer with Python and Databricks
Core
Design, implement, and maintain scalable data pipelines for ingestion, processing, and transformation using cloud services to support data scientists and analysts.
Role type
Senior Data Engineer
Builds
Data pipelines and data solutions in Azure
Domain
Cloud data engineering, Azure ecosystem
Required skills
Python, SQL, Azure Data Factory, Azure Synapse, Databricks/Spark, Polars, Pandas, ETL/ELT, Data Warehousing, Relational Databases, NoSQL Databases, Event-Driven Architecture
Preferred skills
Data validation and cleansing, Data monitoring and observability, Kimball and Star Schema, Agile methodologies
Technologies
Azure, Data Factory, Synapse, Databricks, Spark, Polars, Pandas, Fabric