DATA ENGINEER JAVA SPARK H/F
Core
Building, optimizing, and stabilizing data pipelines for the DatahubV2 platform within a modern data architecture.
Role type
Senior Data Engineer (Java/Spark)
Builds
Scalable batch and distributed data processing pipelines, Spark as a Service on Kubernetes, and data orchestration workflows.
Domain
Big Data, Cloud Infrastructure, Data Engineering
Deliverable
production ML models | product features
Required skills
Java, Apache Spark, Kubernetes, SQL, Python, CI/CD pipelines, Log analysis
Preferred skills
Scala, Trino, Airflow/Astronomer, Banking domain experience
Technologies
Spark 3, Kubernetes, Trino, Airflow, Astronomer, Gitlab, Jenkins, ArgoCD, ELK, Vault, S3
Responsibilities
Design and optimize batch and distributed data pipelines; Develop Spark 3 treatments on Kubernetes; Manipulate data via Trino SQL; Orchestrate data workflows with Airflow/Astronomer; Ensure data job quality and performance; Contribute to DevOps pipeline maintenance; Analyze application logs via ELK.
Seniority
Senior, hands-on IC
