Senior Data Engineer - Product
Core
Building and operating the DSF platform to support real-time fraud detection and risk evaluation for financial institutions.
Role type
Senior IC Big Data Engineer (Distributed Systems)
Builds
DSF platform, Spark jobs, Hadoop ecosystem components, data ingestion pipelines, and interactive workflows for data scientists.
Domain
Financial crime prevention / Big Data Engineering
Deliverable
production ML models | infrastructure
Required skills
Apache Spark (tuning, debugging, orchestration), Java, Hadoop ecosystem (HDFS, YARN), Linux, ETL/ELT pipelines, cloud environments, continuous delivery, on-call operations
Preferred skills
EMR or Kubernetes, AWS S3/Glue, Nessie, Trino, Kafka, Iceberg, Airflow, OSS contributions, DS/ML engineering platforms
Technologies
Apache Spark, Java, HDFS, YARN, EMR, Kubernetes, AWS S3, AWS Glue, Nessie, Trino, Kafka, Iceberg, Airflow, JupyterLabs, Firehose
Responsibilities
Re-architect and scale big data processing components; Analyse workload patterns to drive performance and cost improvements; Ensure stability of Spark jobs on EMR or Kubernetes clusters; Operate and evolve Hadoop ecosystem components; Maintain and improve ingestion pipelines; Improve developer experience across JupyterLabs and DS API; Collaborate on DSF roadmap execution; Own services lifecycle with DevOps practices and participate in on-call rotation.
Seniority
Senior, hands-on IC
