Industry Data Research & Validation P16
Core
Design, build, and maintain scalable data pipelines and data solutions on Microsoft Azure to support business analytics, reporting, and advanced data use cases.
Role type
Data Engineer (Azure ecosystem)
Builds
Scalable data pipelines, ETL/ELT workflows, and data modeling solutions
Domain
Cloud data engineering (Microsoft Azure)
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Azure Data Factory, Azure SQL, Azure Data Lake Storage Gen2, Azure Databricks (PySpark/Spark SQL), SQL, Python, PySpark, ETL/ELT design patterns, data warehousing concepts, incremental loads, CDC, data modeling, Azure DevOps, CI-CD, Git
Preferred skills
Power BI, data governance tools (Purview), Agile/Scrum
Technologies
Azure Data Factory, Azure SQL, Azure Data Lake Storage Gen2, Azure Databricks, PySpark, Spark SQL, Python, SQL, Azure DevOps, Git, Power BI, Purview
Responsibilities
Design and develop data pipelines for ingesting, transforming, and loading data; Build and optimize ETL/ELT workflows using Azure-native tools; Ensure data quality, integrity, and accuracy through validation and monitoring; Implement data modeling solutions for analytics and reporting; Collaborate with business analysts, data scientists, and stakeholders to understand data requirements; Optimize performance and scalability of data pipelines and storage; Ensure data governance, security, and compliance standards are met; Troubleshoot data issues and provide solutions proactively.
Seniority
Mid-level (2-4 years experience)
