Data Engineer (m/w/d)
Core
Design, implement, and operate fully automated data pipelines (ETL, ELT) for efficient processing of large volumes of structured and unstructured data using Azure Databricks.
Role type
Data Engineer
Builds
Automated data pipelines and data platforms
Domain
Cloud data engineering (Azure Databricks)
Deliverable
production ML models
Required skills
Python, SQL, Spark, Azure Databricks, ETL/ELT, API integration, database connectivity, data validation and cleansing
Preferred skills
Experience in regulated environments, continuous learning mindset
Technologies
Azure Databricks, Spark, Python, SQL
Responsibilities
Design and implement automated data pipelines; Integrate internal and external data sources; Ensure data quality and consistency; Monitor pipelines and resolve operational issues; Document work results
Seniority
Mid-level, hands-on IC