Data Engineer (Oil&Gas)
Core
Building and maintaining gas sales management and compliance platforms for the oil & gas industry.
Role type
Data Engineer (Oil & Gas)
Builds
Gas sales management and compliance platforms
Domain
Oil & Gas, Energy
Deliverable
production ML models | product features | infrastructure
Required skills
Azure Synapse Analytics, Azure Data Factory, Azure Databricks, PySpark, ETL development, data pipeline creation, data ingestion from structured and unstructured sources, process orchestration
Preferred skills
Azure Stream Analytics, Azure Blob Storage, Azure Data Lake Storage, Azure Event Hubs, Azure Application Insights
Technologies
Azure Synapse, Azure Data Factory, Azure Databricks, PySpark, Azure Blob Storage, Azure Data Lake Storage, Azure Event Hubs, Azure Application Insights
Responsibilities
Develop gas sales management and compliance platforms; maintain existing data pipelines and deliverables; provide operational support for code-based fixes; create new data pipelines; perform transformations and ETL activities in Synapse; ingest data from SQL, XLS, CSV, and SharePoint sources; orchestrate ETL processes with Synapse and Azure Data Factory.