Data Engineer
Core
Designing and maintaining real-time data streaming pipelines and building robust data lake infrastructure to support AI-powered products and financial forecasting.
Role type
Senior Data Engineer (AI/ML infrastructure)
Builds
Real-time operational efficiency platform and financial forecasting initiatives
Domain
Environmental services, AI/ML, Financial forecasting
Deliverable
production ML models | infrastructure
Required skills
Real-time data streaming, Data lake architecture, AWS Glue, Apache Spark, Kafka, Terraform, CloudFormation, Statistical modelling, Feature engineering, Model training infrastructure, Real-time inference pipelines
Preferred skills
AI/ML technology integration
Technologies
AWS Glue, Lambda, Apache Spark, Kafka, Terraform, CloudFormation
Responsibilities
Design, develop, and maintain scalable real-time and batch data pipelines; Implement and evolve data lake architecture; Build and own end-to-end AI and ML workflows; Apply statistical modelling techniques to validate data quality and develop forecasting models; Collaborate with cross-functional teams to translate data requirements into scalable solutions; Optimize and troubleshoot complex data pipelines for high availability and performance.
Seniority
Senior, hands-on IC