Data Engineer
Core
Building and maintaining real-time ETL pipelines, data warehouses (Redshift, BigQuery), and Python-based data services/APIs to drive business insights and user segmentation.
Role type
Senior Data Engineer (ETL & Data Platform)
Builds
Real-time data pipelines, data warehouses, Flask/Postgres services, and ML inference endpoints
Domain
Betting & Prediction Markets
Deliverable
production ML models
Required skills
Python (ETL frameworks like Airflow/Luigi), REST API development, SQL/Postgres, ML model training & deployment, Pandas/Numpy/scikit-learn, Kubernetes, AWS Redshift, Google BigQuery
Preferred skills
Experience with user segmentation, operational anomaly detection, recommender systems
Technologies
Python, Flask/FastAPI, Postgres, AWS Redshift, Google BigQuery, Kafka, Kubernetes, Jenkins, dbt, Prometheus, Grafana, Elastic Search, Kibana, Lightdash
Responsibilities
Develop and maintain real-time ETL pipelines; Ensure data lake and warehouse health; Develop Flask services and Postgres databases; Assist teams with automated reporting pipelines; Train and deploy ML models for user segmentation and anomaly detection
Seniority
Mid-Senior, hands-on IC
