Digital Technology Experienced Professional- Data and Analytics
Core
Senior Data Engineer designing and optimizing big data ETL pipelines, data lakehouse architectures, and predictive analytics solutions for the energy sector.
Role type
Senior IC data engineer (big data/lakehouse)
Builds
End-to-end data applications, ETL pipelines, and predictive analytics solutions for energy customers.
Domain
Energy industry + Big Data Engineering
Deliverable
production ML models | product features | infrastructure
Required skills
Big data ETL pipeline design, Lakehouse architecture, Distributed data processing, Databricks, PySpark, Apache Spark, Delta Lake, SQL/NoSQL data modeling, Git/CICD, Airflow/Oozie, BigQuery, Hive, Object Storage
Preferred skills
Streaming or near real-time data flow design, Exploratory data analysis, Data quality assessment, Novel tool/methodology invention
Technologies
Databricks, PySpark, Apache Spark, Delta Lake, Airflow, Oozie, BigQuery, Hive, NoSQL, Object Storage
Responsibilities
Collaborate with Product Managers and Architects to define data application scope and roadmap; Design and develop ETL pipelines and orchestration for data lakehouse; Research and evaluate new open-source technologies; Integrate domain data knowledge into development; Document best practices and mentor developers; Identify downstream implications of data loads and migrations.
Seniority
Senior, hands-on IC
