Analyst - Data Engineer
Core
Design, develop, and maintain data pipelines and architecture feeding into the organization's data lake to enable data-driven decision-making.
Role type
Senior Data Engineer (Data Architecture & Pipelines)
Builds
Data pipelines, data lake infrastructure, and data products for analytics teams
Domain
Energy / Big Data / Cloud
Deliverable
production ML models | product features
Required skills
Data pipeline design, ETL principles, Data modeling, Cloud data architecture, Data governance, SQL, Scala, Python/PySpark, Azure Data Factory, Azure Data Lake, Databricks, Hadoop, BigQuery, Hive, Impala, Spark, Kafka
Preferred skills
.NET, Tableau, Power BI, Looker, Shiny, Machine Learning concepts, Computer Vision, Statistics, Applied Mathematics
Technologies
Azure Data Factory, Azure Data Lake, Databricks, Scala, Python, PySpark, .NET, SQL, Hadoop, BigQuery, Hive, Impala, Spark, Kafka, Tableau, Power BI, Looker, Shiny
Responsibilities
Design and optimize data architecture and pipelines adhering to ETL principles; Develop complex queries and solutions using Scala, .NET, and Python/PySpark; Implement and maintain data solutions on Azure Data Factory, Azure Data Lake, and Databricks; Define and design the data catalogue for efficient data discovery; Mentor data and analytic professionals on data standards; Lead evaluation and deployment of emerging analytic data engineering tools; Partner with business analysts and solutions architects on technical architectures.
Seniority
Senior, hands-on IC with mentorship responsibilities
