Senior or Staff Software Engineer, Data Platform (Canada)
Core
Build highly available and scalable data infrastructure to analyze blockchain transaction activity at petabyte scale for public and private sector agencies.
Role type
Senior Staff Software Engineer, Data Platform
Builds
Highly reliable data services, complex ETL pipelines, and large database clusters for blockchain analysis.
Domain
Blockchain data infrastructure, distributed systems, petabyte-scale data processing
Deliverable
production ML models
Required skills
Python, SQL, SparkSQL, distributed system architecture, data modeling, database cluster management, ETL pipeline design, data orchestration (Airflow, DBT), stream processing (Spark, Kafka, Flink), cloud infrastructure (Docker, Terraform, Kubernetes, Datadog)
Preferred skills
AI fluency, rapid prototyping, automation of routine tasks
Technologies
Iceberg, Trino, BigQuery, StarRocks, Citus, Airflow, DBT, Spark, Kafka, Flink, Docker, Terraform, Kubernetes, Datadog, pgbouncer
Responsibilities
Design and architect data models for optimal storage and retrieval; develop complex ETL pipelines for real-time data processing; oversee deployment and monitoring of large database clusters; build scalable engines for routine maintenance tasks; collaborate with data scientists and product managers on novel data models.
Seniority
Senior/Staff, hands-on IC with architectural leadership