Intermediate Data Engineer
Core
Design, build, and support scalable data solutions, pipelines, and cloud-based data platforms to enable analytics, reporting, and operational business processes.
Role type
Intermediate Data Engineer
Builds
Data pipelines, ETL/ELT solutions, data models, and cloud-based data platforms using Microsoft Fabric
Domain
Financial Services / Cloud Data Engineering
Deliverable
production ML models | product features
Required skills
SQL, Python, Data Warehousing, Data Modelling, ETL/ELT Development, Data Integration, Cloud Data Platforms, Performance Optimization, Data Quality Management, Source-to-Target Mapping
Preferred skills
Microsoft Fabric, Fabric Data Factory, Fabric Lakehouse, Fabric Data Warehouse, Fabric Notebooks, OneLake, Medallion Architecture, Spark Fundamentals, Generative AI Concepts, AI-Assisted Development Tools, Microsoft Copilot, Prompt Engineering Fundamentals, Data Preparation for Machine Learning, Intelligent Process Automation, Metadata and Data Lineage Concepts
Technologies
Microsoft Fabric, Data Factory, Lakehouse, Data Warehouse, Notebooks, OneLake, Spark
Responsibilities
Design, develop, test, and maintain batch and real-time data pipelines; Build and support ETL/ELT solutions using modern cloud-based technologies; Develop and optimize data ingestion, transformation, and consumption layers; Support data integration across enterprise platforms and source systems; Implement data quality, reconciliation, and validation controls; Assist with production support activities, root cause analysis, and issue resolution; Participate in migration and modernization initiatives involving Microsoft Fabric, cloud data platforms, and enterprise data warehouses; Develop reusable frameworks, automation utilities, and monitoring capabilities that improve operational efficiency; Collaborate with architects and senior engineers on data modelling, platform design, and solution implementation; Contribute to CI/CD implementation, automated testing, and DevOps practices; Work with governance and security teams to ensure compliance with enterprise data management standards; Participate in Agile ceremonies, technical design discussions, and estimation activities; Create technical documentation and support knowledge transfer activities; Design and develop data solutions using Microsoft Fabric including Data Factory, Lakehouse, Data Warehouse, Notebooks, and Real-Time Intelligence capabilities; Develop and optimize ELT pipelines within Microsoft Fabric; Implement Medallion Architecture patterns (Bronze, Silver, Gold) for enterprise data products; Build data models and curated datasets to support analytics and reporting workloads; Collaborate with AI and analytics teams to prepare data for machine learning and generative AI use cases; Develop AI-assisted operational solutions using Microsoft Copilot, Fabric AI capabilities, and intelligent automation frameworks; Support implementation of metadata management, lineage, and data observability capabilities; Participate in pilot initiatives involving AI-driven data quality monitoring, impact assessment, operational intelligence, and self-healing data pipelines.
Seniority
Intermediate, hands-on IC