Lead Data Engineer GFT - Halifax
Core
Lead the design, development, and deployment of scalable data pipelines and infrastructure to support large-scale data processing and analysis for the Enterprise ESG framework.
Role type
Lead Data Engineer (Climate Solutions)
Builds
Scalable data pipelines, micro-service architectures, and production systems integrating analytics and ML models.
Domain
Financial Services / Climate Solutions / Big Data
Deliverable
production ML models | data pipelines | infrastructure
Required skills
Hadoop, Apache Spark, Python, Scala, Hive, Database optimization, CI/CD, Team leadership, Microservices architecture
Preferred skills
Cloud platforms (AWS, Azure, GCP), Data governance, Frontend/Backend engineering, AI/ML experience, Financial industry risk analytics
Technologies
Hadoop, Apache Spark, Python, Scala, Hive, AWS, Azure, GCP, Git
Responsibilities
Design and maintain scalable data pipelines; Contribute to micro-service data pipeline architecture; Collaborate with data scientists and domain experts; Ensure data quality and integrity; Optimize database performance; Support integration of ML models into production.
Seniority
Senior, hands-on IC with team leadership
