Data Engineer bij BeFrank
Core
Building scalable data solutions and transforming raw data from various sources into data products that deliver business value for a pension platform.
Role type
Senior Data Engineer
Builds
Production data pipelines and data products for a pension company's digital platform
Domain
Financial Services / Data Engineering
Deliverable
production ML models | product features
Required skills
Python, PySpark, Delta Lake, Medallion Architecture, CI/CD, Unit Testing, Integration Testing, Data Quality Validation, SQL, NoSQL, REST APIs, SFTP, Azure Data Stack (Synapse, Data Factory, Azure Functions)
Preferred skills
Experience with partitioning, schema evolution, time travel, merge/upsert patterns
Responsibilities
Write production-grade data pipelines using Python and PySpark; Connect data from diverse sources including relational/NoSQL databases and APIs; Design and implement automated tests and data quality validations; Collaborate with analysts on data models; Design technical solutions within the cloud architecture; Contribute to team knowledge sharing
Seniority
Senior, hands-on IC
