Data Engineer
Core
Design, develop, and maintain scalable data pipelines and workflows using Microsoft Azure cloud services to support business objectives.
Role type
Azure Data Engineer
Builds
Scalable data pipelines, data storage solutions, and data transformation processes
Domain
Cloud data engineering (Microsoft Azure)
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Data Lake Storage, Azure Synapse Analytics, SQL, PySpark, Python, Git, data modeling, ETL processes, data warehousing concepts
Preferred skills
Airflow, data integration and orchestration tools, troubleshooting data pipelines
Technologies
Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Data Lake Storage, Azure Synapse Analytics, Airflow, Python, PySpark
Responsibilities
Design and maintain scalable data pipelines and workflows; Implement and optimize data storage solutions; Collaborate with data scientists and analysts to understand data requirements; Ensure data quality, integrity, and security; Optimize performance for real-time and batch processing; Troubleshoot and resolve data pipeline issues; Perform data analysis, validation, and rectification of master data inconsistencies
Seniority
Mid-level (3-5 years experience)
