Senior Data Engineer - Financial Flows
Core
Design, build, and operate resilient batch and real-time streaming data pipelines powering commodity finance analytics and predictive models.
Role type
Senior IC data engineer (financial data infrastructure)
Builds
Production-grade data pipelines, cloud-native services, and calculation engines for market participants
Domain
Commodities, energy, and maritime financial markets
Deliverable
production ML models
Required skills
Python, AWS, Docker, Kubernetes, SQL, time-series databases, streaming data ingestion, data quality frameworks, CI/CD, GitOps
Preferred skills
Financial market domain knowledge, legacy PHP migration, major financial vendor integrations
Technologies
AWS, Docker, Kubernetes, QuestDB, Elasticsearch, PHP, Python
Responsibilities
Design and operate batch and real-time streaming data ingestion pipelines; Lead architectural migration of legacy backend infrastructure to AWS; Refactor calculation blocks from PHP to Python; Ingest and model complex financial datasets; Implement automated data validation and monitoring; Deploy and scale containerized services; Partner with researchers to translate trading models into pipelines; Mentor peer engineers
Seniority
Senior, hands-on IC
