Data Engineer
Core
Design and build scalable data platforms, pipelines, and warehouses to ingest, clean, store, and serve petabyte-scale data for live trading strategies and backtesting.
Role type
Senior Data Engineer (Financial Data Platform)
Builds
Production data warehouse, real-time data feeds, automated data cleansing and monitoring systems, and in-house bitemporal data store
Domain
Financial services / Quantitative trading / Big Data
Deliverable
production ML models
Required skills
Python, SQL, Data Modeling, Unix/Linux, Apache Airflow, PostgreSQL, MongoDB, S3, CI/CD
Preferred skills
Java, Cloud platforms (AWS/GCP), On-premises data infrastructure, Financial industry experience
Technologies
Python, Java, Apache Airflow, Slurm, NATS, PostgreSQL, MongoDB, S3, NFS, GitHub Actions
Responsibilities
Design and onboard new data sources into the warehouse; Build automated systems for data cleansing, anomaly detection, and alerting; Evaluate and implement new tools for organizing and streaming large datasets; Support the production data warehouse; Develop and maintain vendor relationships for data sources
Seniority
Senior, hands-on IC