Senior Data Platform Engineer
Core
Building foundational tooling and processes for on-premise Big Data Platforms supporting 10s of products and 1000s of daily jobs.
Role type
Senior Data Platform Engineer
Builds
Scalable big data platforms, distributed systems, and data processing frameworks for internal data engineers and ML scientists.
Domain
Financial technology / Big Data Infrastructure
Deliverable
infrastructure
Required skills
Python (object-oriented), distributed data systems (Spark, Trino, Druid), DevOps pipelines, real-time/batch data pipelines (Kafka, Spark streaming), Kubernetes/Hadoop ecosystems, metadata management, streaming processing frameworks.
Preferred skills
Experience with large-scale private cloud systems, data modeling tooling, continuous release environments.
Technologies
Spark, Trino, Druid, Kafka, Kubernetes, Docker, Hadoop, Hive, Yarn, HDFS, Kerberos
Responsibilities
Develop and maintain scalable high-performance big data platforms; work with distributed systems (databases, filesystems, compute); improve continuous release and deployment environments; build tools for data discoverability and quality; introduce technologies to reduce time to insight; develop streaming processing applications.
Seniority
Senior, hands-on IC
