SENIOR DATA ENGINEERING ANALYST (36253)
Core
Design and maintain data pipelines to extract structured and delta data from SAP source systems into a cloud data lake, implementing Medallion Architecture (Bronze, Silver, Gold) to deliver business-ready data products.
Role type
Senior Data Engineering Analyst
Builds
Data pipelines ingesting SAP data into Azure Data Lake Storage and Azure Databricks
Domain
Enterprise Data Engineering / SAP Integration
Deliverable
production ML models
Required skills
SQL, Python, Spark/PySpark, SAP data extraction (CDS Views, ODP/ODQ, SLT, APIs), Azure Data Lake Storage, Azure Databricks, Delta Lake, Parquet, Medallion Architecture, pipeline orchestration (Azure Data Factory, Databricks Workflows)
Preferred skills
Git, CI/CD, automated testing, observability, Infrastructure as Code, SAP BW, SAP Datasphere, SAP HANA, data governance tools
Technologies
Azure Data Lake Storage, Azure Databricks, Delta Lake, Parquet, Azure Data Factory, Databricks Workflows, SAP CDS Views, SAP ODP/ODQ, SAP SLT
Responsibilities
Assess SAP source systems and define secure extraction strategies; Develop and maintain data pipelines ingesting data from SAP into the data lake; Design and implement Medallion Architecture; Implement incremental loads, change data capture, orchestration, monitoring, and data reconciliation; Ensure data quality, lineage, security, and compliance; Optimize pipeline performance and costs; Collaborate with stakeholders to translate requirements into reusable data products.
Seniority
Senior, hands-on IC