Data Engineer

## About the role Provide support in designing and overseeing enterprise-grade data pipelines and data stores, essential for the development of advanced analytics programs, machine learning models, and statistical methods. ## Responsibilities - Implement automation and streamline processes to optimize the entire data and analytics platform, ensuring efficient throughput and high-performance outcomes. - Recognize, devise, and execute internal process enhancements, including automation of manual tasks, optimizing data delivery, and redesigning architecture or infrastructure to enhance scalability. - Collate large, intricate datasets that align with functional and non-functional business demands. - Develop processes that facilitate data transformation, manage data structures, metadata, dependencies, and workload management. - Collaborate with business users to understand functional and data requirements, contributing to the enhancement of data models and pipelines. - Design, develop, and maintain pipeline solutions utilizing Microsoft Azure data warehouse and/or (Synapse). - Work on various aspects of data warehouse/Synapse/ADF, including Pipelines, Dataflows, Notebooks (Python), Gen 2 Storage, and SQL Serverless/Dedicated Pool. - Apply a deep understanding of data warehousing principles and ETL processes. - Develop interactive dashboards using Microsoft Power BI, incorporating drill-down data exploration for insightful analytics. - Conduct proficient SQL querying and contribute to data modeling efforts. - Adhere to data visualization best practices and user experience design principles ## Required Qualifications - Bachelor's Degree in Computer Science, Information Systems, or a related field. - 3+ years of experience as a data engineer, BI developer or in a similar role, leveraging Microsoft technologies. - Hands-on experience with Microsoft Azure services, such as Azure Data Factory, data warehouse, Azure Analysis Services, or Azure Synapse Analytics. - Exhibit a thorough understanding of Data Lake architectures, including raw, enriched, and curated layer concepts, and ETL/ELT operations. - Exhibit a solid understanding of database design, data warehousing concepts, big data platforms, and ETL/ELT operations. - Experience working with data integration techniques & self-service data preparation. - Experience in requirements analysis, design, and prototyping. - Experience deploying modern data solutions leveraging components like Azure functions, Azure Data Factory, Data Flows, Azure Data Lake, Azure SQL, Azure Synapse, Streaming Analytics or equivalent on another cloud provider such as AWS or Google Cloud. - Experience with DevOps tools like Azure DevOps, Jenkins, Maven etc. - Experience in building/operating/maintaining fault tolerant and scalable data processing integrations. - Demonstrated experience of turning business use cases and requirements into technical solutions. - Strong level of understanding on Azure Data Factory, SQL/Synapse, ADLS, and Azure DevOps. - Ability to conduct data profiling, cataloging, and mappings for technical design and construction of data flows.