Data Engineer (m/w/d)
Core
Design, implement, and operate fully automated data pipelines (ETL, ELT) for efficient processing of large volumes of structured and unstructured data using Azure Databricks.
Role type
Data Engineer
Builds
Automated data pipelines and data platforms
Domain
Data Engineering / Cloud Data Platform
Deliverable
production ML models | infrastructure
Required skills
Python, SQL, Spark, Azure Databricks, ETL/ELT pipeline development, database integration, API integration, data validation and cleansing processes, data quality assurance, data security and compliance in regulated environments
Preferred skills
Experience with web crawlers, continuous learning mindset
Technologies
Azure Databricks, Spark, Python, SQL
Responsibilities
Design and implement automated data pipelines; integrate internal and external data sources; ensure data quality and consistency; monitor pipelines and resolve operational issues; document work results
Seniority
Mid-level, hands-on IC
