Senior Data Engineer (m/w/d)
Core
Designing and developing an on-premises data platform with data orchestration, deployment, and platform automation components.
Role type
Senior IC data engineer (on-premises platform)
Builds
On-premises data platform, batch and streaming pipelines, platform services
Domain
Data engineering, cloud-native infrastructure, on-premises data management
Required skills
Kubernetes, Linux, Python, Golang, SQL (MPP), Bash, Java, batch and streaming pipelines, data catalog integration, data versioning
Preferred skills
Proof of Concepts with AI-agent tools (e.g., Cursor.ai), NUMA and Core Pinning concepts, edge data management without public cloud
Technologies
Kubernetes, S3, Apache Iceberg, GitOps, NATS.io, Risingwave, Dagster, Spark, Kafka, Flink, Airflow, Cursor.ai
Responsibilities
Develop and integrate components for data orchestration, deployment, and platform automation; implement batch and streaming data pipelines; integrate workloads on Kubernetes and S3-compatible storage; investigate data catalogs and their integration into the platform; create proofs of concepts using AI-agent tools.
Seniority
Senior, hands-on IC