Data Engineer
Core
Design, build, and maintain scalable, automated, fault-tolerant data pipelines and integration solutions for enterprise analytics and strategic business use.
Role type
Senior Data Engineer
Builds
Data pipelines, data integration solutions, and analytics environments
Domain
Enterprise Data Engineering, Cloud Data Warehousing
Deliverable
production ML models
Required skills
SQL, ETL/ELT design, Data pipeline architecture, Data profiling, Metadata management, Data quality monitoring, Code review, Unit testing
Preferred skills
Event store and stream processing (Apache Kafka, Confluent), API development (MuleSoft, Axway), Data virtualization
Technologies
IBM InfoSphere DataStage, Oracle Data Integrator, Informatica PowerCenter, MS SSIS, AWS Glue, Denodo, Snowflake, Netezza, Teradata, Redshift, Apache Kafka, Confluent, MuleSoft, Axway
Responsibilities
Create data mappings, programs, routines, and SQL to acquire data from legacy, web, cloud, and purchased package environments; Profile data to assess accuracy and completeness; Monitor data quality to detect emerging issues and create transformation rules; Maintain metadata management processes and documentation; Participate in code reviews and unit testing to optimize performance.
Seniority
Mid-Senior, hands-on IC
