DATA ENGINEER JAVA SPARK H/F
Core
Building, optimizing, and stabilizing data pipelines for the DatahubV2 platform using Java, Spark, and modern data architectures.
Role type
Senior Data Engineer (Java/Spark)
Builds
Distributed data pipelines, Spark 3 services on Kubernetes, and data workflows for the DatahubV2 platform.
Domain
Big Data / Cloud Infrastructure
Deliverable
production ML models
Required skills
Java, Spark 3, Kubernetes, SQL, Python, Apache Airflow, DevOps (CI/CD)
Preferred skills
Scala, banking context experience
Technologies
Spark 3, Kubernetes, Starburst (Trino), Astronomer, Apache Airflow, ELK, Gitlab, Jenkins, ArgoCD
Responsibilities
Design and optimize batch and distributed data pipelines; Develop Spark 3 treatments on Kubernetes; Manipulate DatahubV2 data via Starburst (Trino); Build and orchestrate data workflows on Astronomer/Airflow; Ensure data job quality, performance, and reliability; Contribute to DevOps pipeline maintenance.
Seniority
Senior, hands-on IC