Full stack Software Engineer
Core
Design, develop, and maintain scalable data pipelines and ETL processes using Microsoft Azure services to create resilient data products.
Role type
Senior Data Engineer (Full Stack focus on Data Engineering)
Builds
Data pipelines, ETL processes, and data products
Domain
Energy sector, Cloud Data Engineering
Deliverable
production ML models
Required skills
Azure Data Factory, Azure Synapse, Azure Databricks, SQL (SQL Server, PostgreSQL), PySpark, CI/CD, Git, Azure DevOps, Ansible, Parquet, Delta, Avro
Preferred skills
Python, Scala, Java, C#
Technologies
Microsoft Azure, Azure Data Lake Storage Gen 2, Azure Blob Storage, Spark, SQL Server, PostgreSQL
Responsibilities
Design and maintain data pipelines and ETL processes; Optimize pipelines for performance and scalability; Collaborate with data scientists and analysts; Monitor and troubleshoot pipeline issues; Manage CI/CD deployment processes; Develop documentation for data processes.
Seniority
Senior, hands-on IC