Data Engineer
Required skills
Strong Python, Strong SQL, Practical experience with cloud data platforms, Hands-on experience with streaming systems, Familiarity with Airflow, CI/CD, DevOps practices, Experience with Docker, Kubernetes, and Infrastructure-as-Code, Understanding of data contracts, time-series data, and event-driven systems, Fluency in English
Preferred skills
Interest in the energy domain
Technologies
Azure, Databricks, Python, Kafka, Spark Structured Streaming, Flink, Airflow, Docker, Kubernetes, Infrastructure-as-Code
Responsibilities
Design, build, and operate batch and real-time data pipelines, Ingest and process external and internal data, Work with streaming technologies, Ensure high data quality through validation, monitoring, and lineage, Develop and improve data workflows, standards, and reliability in production systems, Enable forecasting, analytics, and ML use cases, Build Kafka based central streaming data platform from scratch, Operate and continuously enhance the components once the platform is in production
Seniority
Experienced
Domain
Energy