Senior Data Engineer (Databricks)
Core
Design, operate, and improve an end-to-end SAP data replication platform streaming change data from SAP S/4HANA/ECC into Databricks for analytics, finance, supply chain, and operations.
Role type
Senior Data Engineer (Streaming/CDC)
Builds
Real-time data pipelines from SAP to Databricks using Kafka/Confluent and Azure infrastructure
Domain
Enterprise data engineering, SAP ecosystem, Cloud (Azure)
Deliverable
production ML models | product features | dashboards & analysis | infrastructure
Required skills
CDC pipeline engineering, Apache Kafka/Confluent, Python, SQL (HANA/Spark/T-SQL), Kubernetes/AKS, Terraform/Bicep, Linux administration, CI/CD, Observability (Grafana/Prometheus/ELK)
Preferred skills
HVR/Debezium, Databricks/Delta Lake, ksqlDB/Flink, Snowflake, Scala/Java, SAP functional knowledge
Technologies
Databricks, Delta Lake, Unity Catalog, Kafka, Confluent, HVR, Debezium, Azure (AKS, Key Vault, Event Hubs), Kubernetes, Terraform, Bicep, GitHub Actions, Azure DevOps, Grafana, Prometheus, ELK, Splunk, Snowflake
Responsibilities
Design and build pipelines from SAP to downstream systems; Develop and maintain Kafka/Confluent topics, connectors, schemas, and stream-processing jobs; Build automation for deployment, validation, and reconciliation; Deploy and manage Kafka/HVR components on AKS/Kubernetes; Maintain CI/CD pipelines and IaC; Enforce security, RBAC, and network segmentation; Author runbooks and architecture diagrams
Seniority
Senior, hands-on IC
