Data Engineering Lead
Core
Lead data acquisition function by defining ingestion strategy and building Azure-based data pipelines for Trading, Risk, Finance, and Operations.
Role type
Senior Data Engineering Lead
Builds
Azure-based data pipelines and platform solutions
Domain
Financial services / Energy trading
Deliverable
production ML models | infrastructure
Required skills
Azure data services (ADLS, ADF, Databricks, Synapse, Event Hub/Kafka), Python, SQL, Spark/PySpark, data modeling, metadata and lineage tooling, performance & cost optimization, CI/CD, test automation
Preferred skills
Azure Data Engineer or Databricks certifications, experience with ETRM/Endur, market data (ICE, Platts, Bloomberg), SAP
Technologies
Azure, ADLS, ADF, Databricks, Synapse, Event Hub, Kafka, Python, SQL, Spark, PySpark, SAP
Responsibilities
Own ingestion strategy and define scalable batch/streaming data patterns; Design and deliver Azure-based data pipelines and platform solutions; Standardize connectors, schemas, controls, CI/CD, IaC, and automated testing; Lead onboarding of enterprise and market data sources with SLAs and observability; Act as domain Data Manager for lineage, metadata, quality, access, and master data ownership; Ensure compliance with data lifecycle, audit, records management, and governance standards; Manage capacity planning, skills development, and vendor contributions
Seniority
Senior, hands-on IC with leadership
