Senior Data Engineer

## About the role Design and implement data integration procedures and pipelines that extract data from various sources, transform it into the desired format, and load it into the appropriate modern analytics data storage and management systems. Integrates data from-to different internal and external sources (batch, incremental, streaming). Adoption and drive of active metadata usage in data integration processes with high level of automation and simplicity. You will be responsible for using innovative and modern tools, techniques, and architectures to automate the most-common, repeatable, and tedious data preparation and integration tasks partially or completely. Collaborates with analytics owners (business analysts, project finance analysts, domain owners, and SMEs) to optimize data products in domain of data and business intelligence responsibility. Improving data quality and governance with business data owners. Educate and train counterparts in these data pipelining and preparation techniques, which make it easier for them to integrate and consume the data they need for their own use cases. Ensures data consistency and integrity during the integration process, identifying root cause of quality issues, address them and work with technical system owners to identify and implement optimal solution. Optimizes data pipelines and data processing workflows for performance, scalability, and efficiency. Monitors and tunes data analytics systems, identifies and resolves performance bottlenecks, and implements caching and indexing strategies to enhance query performance. Implements data quality checks and validations (business rules) within data pipelines to ensure the accuracy, consistency, and completeness of data. Takes authority, responsibility, and accountability for exploiting the value of enterprise information assets and of the analytics used to render insights for decision making automated decisions and augmentation of human performance. Establishes the governance of data and algorithms used for analysis, analytical applications, and automated decision making. ## Responsibilities - Design and implement data integration procedures and pipelines - Integrate data from-to different internal and external sources (batch, incremental, streaming) - Adoption and drive of active metadata usage in data integration processes with high level of automation and simplicity - Use innovative and modern tools, techniques, and architectures to automate data preparation and integration tasks - Collaborate with analytics owners to optimize data products in domain of data and business intelligence responsibility - Improve data quality and governance with business data owners - Educate and train counterparts in data pipelining and preparation techniques - Ensure data consistency and integrity during the integration process - Optimize data pipelines and data processing workflows for performance, scalability, and efficiency - Monitor and tune data analytics systems, identify and resolve performance bottlocks, and implement caching and indexing strategies - Implement data quality checks and validations (business rules) within data pipelines - Take authority, responsibility, and accountability for exploiting the value of enterprise information assets and analytics - Establish governance of data and algorithms used for analysis, analytical applications, and automated decision making ## Requirements Qualifications Skills Strong experience ## Nice to Have ## Benefits