Staff Software Engineer, Batch and Realtime Streaming
Core
Architecting, developing, and maintaining a Batch and Realtime Streaming platform that serves as the data backbone for machine learning research and production trading systems.
Role type
Staff Software Engineer (Data Infrastructure & Streaming)
Builds
High-performance data ingestion pipelines, storage systems, serving layers, and API interfaces for feature productization.
Domain
Quantitative Finance / Machine Learning Infrastructure
Deliverable
production ML models | infrastructure
Required skills
Distributed computing, data infrastructure, Python, Go, database technologies (Postgres, MySQL, Cassandra, DuckDB), REST/gRPC APIs, data quality & lineage, pipeline optimization, technical mentorship
Preferred skills
Large-scale data pipelines (Airflow, Spark, Ray, Iceberg), Python data science tooling (pandas, polars, dask), monitoring & observability (Prometheus, Grafana, ELK), feature stores (Feast, Hopsworks)
Technologies
Python, Go, Postgres, MySQL, Cassandra, DuckDB, Airflow, Spark, Ray, Iceberg, Prometheus, Grafana, ELK Stack, Feast, Hopsworks
Responsibilities
Architect and implement core components for batch and realtime streaming platforms; collaborate with research and trading teams to design high-performance interfaces; ensure data quality, consistency, and lineage; optimize pipelines for scalability and reliability; drive adoption of the feature store; mentor junior engineers.
Seniority
Staff, hands-on IC with strategic impact