Senior Data Engineer
Core
Design, develop, and maintain scalable data pipelines and transformations to support BI, analytics, and data science use cases for global commodities markets.
Role type
Senior Data Engineer (Data Platform)
Builds
Modern data platform, data pipelines, and transformations for price reporting and intelligence services.
Domain
Commodities markets (agriculture, metals, mining, energy) / Cloud Data Engineering
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Advanced SQL (Snowflake, MSSQL), Python, dbt, Snowflake (Snowpark), ELT/ETL tools (Fivetran, Airbyte), Data orchestration (Dagster, Airflow), CI/CD, Cloud services (Azure, AWS, GCP), Data modeling (Star Schema, Data Vault)
Preferred skills
Kubernetes, Infrastructure as Code (Terraform), Event streaming (Kafka), AI-assisted development workflows
Technologies
Snowflake, dbt, Python, Dagster, GitHub Actions, Kubernetes, Terraform, Kafka, Azure, AWS, GCP
Responsibilities
Design and maintain scalable data pipelines from diverse sources; Transform data using dbt across layers; Write custom Python connectors; Embed automated data quality checks and alerting; Manage CI/CD workflows; Implement RBAC policies; Leverage AI tools for code generation and review.
Seniority
Senior, hands-on IC