Data Engineer Senior - Data Products Organization (Cloudera Hadoop)
Core
Senior Data Engineer leading the design, build, and maintenance of scalable data solutions and infrastructure for business-critical applications in financial services.
Role type
Senior Data Engineer Lead (Big Data)
Builds
Data integration processes, data service infrastructure, and data solutions for multiple applications.
Domain
Financial Services / Big Data
Deliverable
production ML models | product features | infrastructure
Required skills
Spark/Pyspark, Cloudera Hadoop (HIVE, HQL, Impala, Beeline, HDFS), Linux, Kafka streaming, CA7 orchestration, Data Architecture, Data Migration, Data Transformation
Preferred skills
Data Analytics, Data Mining, Machine Learning (ML), Analytical Thinking
Technologies
Cloudera Hadoop, Spark 3, Python, Kafka, CA7, Linux, Hive, HQL, Impala, Beeline, HDFS, Iceberg
Responsibilities
Act as an internal consultant to enhance code quality and adherence to engineering best practices; Design, build, and test data integration processes between business-critical applications; Review architecture to identify areas for automation, optimization, and cost reduction; Lead data requirement analysis and data preparation process development; Design and build data service infrastructure on multiple data platforms; Consult on data migration and transformation to ensure accuracy and security.
Seniority
Senior, hands-on IC with leadership responsibilities