Data Platform Engineer
Core
Designing and maintaining scalable data pipelines and infrastructure for public health and food security analytics in Sub-Saharan Africa.
Role type
Mid/Senior Data Platform Engineer (Backend)
Builds
Data pipelines, data layers (raw, cleaned, curated), and supporting infrastructure for surveillance and nutrient analysis.
Domain
Public health, food security, and data infrastructure in Sub-Saharan Africa.
Deliverable
production ML models | product features | infrastructure
Required skills
ETL/ELT pipeline design, data pipeline reliability (idempotency, backfills), layered data system architecture, batch/micro-batch/streaming trade-offs, software engineering fundamentals, UNIX/Darwin development.
Preferred skills
Python, Django, DBT, Dagster, Luigi, Clickhouse, containerization (Docker, Kubernetes, Nix), data warehousing, OLAP, metadata management, dimensional modeling, relational database theory.
Technologies
Python, Django, DBT, Dagster, Luigi, Clickhouse, Docker, Terraform, Kubernetes, Nix
Responsibilities
Designing and maintaining data pipelines using ETL/ELT approaches, structuring data systems into clear layers, deciding between batch/streaming approaches based on latency and complexity, participating in design sessions and code reviews.
Seniority
Mid/Senior, hands-on IC
