Data Integration Engineer (m/w/d) – Schwerpunkt Streaming & Confluent
Core
Building a central data and integration platform (Single Point of Truth) for a food and non-food wholesale group, focusing on real-time streaming and event-driven architecture.
Role type
Data Integration Engineer (Streaming & Confluent)
Builds
Central data platform integrating services across the corporate group, specifically Confluent Cloud as the data hub.
Domain
Food & Non-Food Wholesale / Data Engineering / Cloud
Deliverable
production ML models
Required skills
Confluent Cloud (Apache Kafka), Change Data Capture (CDC), Azure Databricks, Python (PySpark), Data Modeling (Data-Vault), ELT processes, Cloud technologies (Azure), Security & Compliance (GDPR)
Preferred skills
Schema Registry, Avro, Debezium, Custom Connector development, Consumer-Lag monitoring, Pipeline monitoring
Technologies
Confluent Cloud, Azure Databricks, Apache Kafka, Python, PySpark, Azure
Responsibilities
Connect data sources to Confluent Cloud via CDC and custom connectors; Design, standardize, and automate integration processes; Operate and maintain integration pipelines; Implement security and data privacy measures; Build reporting and monitoring components; Support data modeling and Lakehouse integration.
Seniority
Mid-level, hands-on IC
