IT engineer data lakehouse - Sales & Finance
Core
Design, develop, and operate scalable data pipelines and data products in Azure Databricks to enable data-driven decision-making for Sales & Finance.
Role type
Senior Data Engineer (Data Lakehouse)
Builds
Scalable batch and streaming pipelines, bronze/silver/gold data layers, and curated data marts for Sales & Finance.
Domain
Automotive industry data engineering and analytics
Deliverable
production ML models | product features | dashboards & analysis
Required skills
PySpark, Scala, Azure Databricks, CI/CD automation, TDD, dimensional modeling, master data management, data quality rule implementation, performance tuning, legacy code refactoring
Preferred skills
Experience with SAP data ingestion, Unity Catalog, semantic models, cross-functional collaboration with ML/AI engineers
Technologies
Azure Databricks, PySpark, Scala, Power BI, SAP, Unity Catalog
Responsibilities
Design scalable batch and streaming pipelines; implement ingestion from structured and semi-structured sources; build lakehouse layering architecture; develop automated data validation tests; monitor pipeline health and optimize cluster resource usage; refactor legacy pipelines; document pipeline logic and technical decisions; align designs with governance and metadata standards.
Seniority
Mid-Senior, hands-on IC
