Senior Data Engineer (TOP Organization)
Core
Design and develop cloud-based data and analytics platforms, implementing ELT/ETL jobs, data orchestration, transformation, and visualization for the Modern IT Service & Asset Management team.
Role type
Senior Data Engineer (Cloud & Data Platform)
Builds
Integrated data pipelines, data warehouses/lakes, and BI dashboards for IT service management.
Domain
Consumer Goods / IT Infrastructure / Cloud Data Engineering
Deliverable
production ML models | product features | dashboards & analysis
Required skills
PySpark, Python, SQL, Azure Data Factory, Azure Databricks, Azure Data Lake Storage, Databricks DeltaLake, Azure Synapse Analytics, Apache Spark, Power BI, DevOps tools (Azure DevOps, Chef, Puppet, Github), Data Modeling, Data Warehousing concepts, Data Governance principles
Preferred skills
Experience with on-premises to cloud data integration, performance optimization of large datasets, semantic data modeling
Technologies
Microsoft Azure, Databricks, Apache Spark, SQL Server, Power BI, Azure DevOps, Chef, Puppet, Github
Responsibilities
Develop and maintain data pipelines for ingestion from various sources; Design and implement data transformation processes including cleansing and normalization; Develop and maintain data models and schemas; Design and build data warehouses or data lakes; Integrate data from multiple on-premises and cloud-based sources; Implement data governance practices for security and compliance; Optimize data pipelines and queries for performance; Monitor data pipelines and troubleshoot issues; Build BI reports for decision making; Document data engineering processes and system configurations
Seniority
Senior, hands-on IC
