Market Data Engineer (Domain Trading Expertise Required)
Core
Design, build, and maintain a petabyte-scale lakehouse infrastructure for capturing, normalizing, and serving market data from raw exchange feeds to support research, backtesting, and real-time trading.
Role type
Senior Market Data Engineer (Lakehouse & Feed Ingestion)
Builds
Unified canonical market data models, L2/L3 order-book reconstructions, and shared data layers for quant research and trading.
Domain
Financial Markets / Algorithmic Trading / Data Engineering
Deliverable
production ML models | product features | infrastructure
Required skills
Apache Iceberg, market data protocols (ITCH, FIX, UDP), packet capture (pcap), Python, Rust, Apache Spark, Airflow, Kubernetes, AWS S3, SQL, Linux, Terraform, Grafana, Prometheus
Preferred skills
Polars, PySpark, Arrow, schema evolution, time-series at scale, multi-vendor normalization (Bloomberg, Refinitiv, ICE)
Responsibilities
Decode and normalize raw exchange feeds into a unified canonical model; Build batch and stream pipelines for tick and reference data; Own L2/L3 order-book reconstruction with gap handling; Design and manage Apache Iceberg lakehouse partitioning and compaction; Build libraries for data contracts and lineage; Embed monitoring, alerting, and SLAs across capture and pipeline layers.
Seniority
Senior, hands-on IC