Data Engineer
Core
Design, build, and maintain robust ETL/ELT pipelines and data models on Azure to deliver trusted datasets for internal and external stakeholders in the ETF industry.
Role type
Senior Data Engineer
Builds
Data pipelines and data models for an ETF analytics platform
Domain
Financial data / Exchange Traded Funds (ETFs)
Deliverable
production ML models
Required skills
Azure Data Factory, Azure Databricks, PySpark, .NET (C#), SQL, Python, Azure Data Lake, Azure Synapse, Azure Cosmos DB, data governance, pipeline monitoring
Preferred skills
Experience with financial data, real-time data processing
Technologies
Azure, Databricks, PySpark, .NET, C#, Python, SQL, Azure Data Factory, Azure Data Lake, Azure Synapse, Azure Cosmos DB, Azure Monitor, Log Analytics
Responsibilities
Design and build ETL/ELT pipelines; Optimize data workflows for performance and cost; Ensure data integrity and quality; Collaborate on real-time data processing; Implement data governance and security best practices; Monitor and troubleshoot pipelines
Seniority
Senior, hands-on IC