Data Engineer (x4)
Core
Designing and constructing scalable data pipelines, lakes, and warehouses to support enterprise analysis, reporting, and business intelligence.
Role type
Senior Data Engineer
Builds
Production data infrastructure (ELT/ETL pipelines, data lakes, data warehouses)
Domain
Energy sector, Cloud Data Engineering
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Python, PySpark, SparkSQL, SQL, Azure Data Factory, Azure Data Lake Storage, Synapse Analytics, Databricks, Data Governance, Data Quality, Dimensional Modeling, CI/CD, Unit Testing, Regression Testing, Code Reviews
Preferred skills
Experience with XML, CSV, JSON, REST APIs, Microsoft Purview, Power BI, SCRUM, Kanban
Technologies
Azure Data Factory, Azure Data Lake Storage, Synapse Analytics, Databricks, Python, PySpark, SQL, Power BI, Microsoft Purview
Responsibilities
Design and implement scalable ELT/ETL pipelines and data infrastructure; Create reliable, maintainable, and high-performance data lakes and data warehouses; Partner with Data Modelers and Architects to establish curated, business-centric data models; Clean, prepare, and optimize datasets for enhanced performance; Diagnose and resolve ingestion, transformation, pipeline performance, and data integrity challenges; Collaborate with analysts, scientists, engineers, and architects to create pipelines for enterprise data sharing; Oversee production solutions, troubleshoot issues, and provide Tier 2 support for datasets; Design and conduct automated unit and regression tests; Participate in peer code reviews to uphold quality and maintainability standards.
Seniority
Senior, hands-on IC
