Software Engineer Data Platform
Core
Building a modern data platform to unlock data from dozens of sources and convert it into products delivering direct value to customers and colleagues.
Role type
Senior Software Engineer (Data Platform)
Builds
Production-grade Python applications, data pipelines, and automated processes on a scalable cloud architecture.
Domain
Financial Services / Data Engineering
Deliverable
production ML models | product features | infrastructure
Required skills
Python, PySpark, Delta Lake, Medallion Architecture, CI/CD, Unit/Integration/BDD Testing, Data Quality Validation, Cloud Architecture, NoSQL/Relational Databases, S3/Blob Storage, REST APIs, SFTP
Preferred skills
Azure Data Stack (Synapse, Data Factory, Azure Functions), Backend Development, Platform Engineering, AI Engineering
Technologies
Python, PySpark, Delta Lake, Azure DevOps, Azure Synapse, Azure Data Factory, Azure Functions, NoSQL, S3, REST, SFTP
Responsibilities
Write production-grade datapipelines in Python and PySpark following the medallion-architecture; Unload data from various source systems including relational/NoSQL databases, Blob/S3 storage, REST APIs, and SFTP servers; Design and implement data quality validations to signal and prevent data issues; Write tests at multiple levels (unit, integration, BDD) and integrate them into the CI/CD process; Collaborate with data and business analysts on data models and data products; Independently design and implement technical solutions within the software/cloud architecture.
Seniority
Senior, hands-on IC
