Data Engineer
Core
Designing, optimizing, and managing large-scale real-time ETL pipelines to integrate corporate banking data into a data lake.
Role type
Senior Data Engineer
Builds
Data pipelines and data lake integrations for corporate banking analytics
Domain
Corporate Banking / Big Data
Deliverable
production ML models | infrastructure
Required skills
ETL pipeline development, SQL, Python, Azure Data Factory, Azure Databricks, Cloudera/Hadoop ecosystem (HDFS, Hive, Spark), data governance, data quality
Preferred skills
Real-time data processing, machine learning models and tools (Azure ML)
Technologies
Azure Data Factory, Azure Databricks, Apache Spark, Cloudera, Hadoop, HDFS, Hive
Responsibilities
Design and optimize ETL pipelines to integrate corporate banking data into the data lake; Integrate data from multiple sources and transform it for analytics using Apache Spark and Databricks; Automate data processes and optimize pipeline performance; Implement data quality checks to ensure accurate and reliable data; Work with business teams and IT to design and document data solutions
Seniority
Senior, hands-on IC
