Associate, Data Engineer, Transformation and Data
Core
Develop, enhance, and support data extraction, transformation, and validation processes to ensure datasets are accurate and fit for consumption by downstream analytics platforms.
Role type
Associate Data Engineer (ETL & Pipeline)
Builds
Batch and ad-hoc data pipelines using Sparkola and PySpark
Domain
Consumer Banking / Financial Services
Deliverable
production ML models | product features
Required skills
Sparkola, PySpark, SQL (complex queries, joins, performance tuning), MS Excel (data validation, analysis, reconciliation)
Preferred skills
Banking or financial services experience, data governance, audit, lineage, and controls
Technologies
Sparkola Job Server, Cloudera Machine Learning (CML), QlikSense, Superset, JIRA
Responsibilities
Develop and support data extraction, transformation, and validation processes; Build and manage batch and ad-hoc data pipelines; Engage with business users to translate functional requirements into technical logic; Support production issues and data clarifications
Seniority
Associate, hands-on IC