Data Architect - Data Engineering 4D
Core
Design, develop, and manage enterprise-grade real-time data pipelines and event-driven integration patterns using Apache Kafka and Confluent Platform.
Role type
Senior Data Architect (Real-time Data Engineering)
Builds
Scalable, fault-tolerant messaging infrastructure and CDC pipelines supporting 29+ source systems
Domain
Enterprise Data Engineering / Real-time Streaming
Deliverable
production ML models | infrastructure
Required skills
Apache Kafka, Confluent Platform, Change Data Capture (CDC), Debezium, Event-driven architecture, Distributed systems, Java, Python, REST APIs, Avro, Protobuf, Linux, Docker, Kubernetes, CI/CD, Prometheus, Grafana
Preferred skills
Confluent Control Center, Apache Flink, Kafka Streams, KSQLDB, Data lakehouse platforms (Databricks, Snowflake, Delta Lake), Cloud platforms (Azure, AWS, GCP)
Technologies
Apache Kafka, Confluent Platform, Debezium, SAP CDC, Docker, Kubernetes, Prometheus, Grafana, Azure, AWS, GCP
Responsibilities
Design scalable event-driven integration patterns; Configure and manage Kafka topics, partitions, replication, and security; Build and maintain CDC pipelines; Deploy and optimize Enterprise Bus consumers and producers; Monitor and optimize Kafka cluster performance; Implement fault-tolerant messaging and disaster recovery strategies; Collaborate with architects and stakeholders to define messaging patterns; Develop automation for deployment and monitoring using CI/CD and IaC.
Seniority
Senior, hands-on IC