Data Engineer 2
Core
Designing and maintaining data structures, pipelines, and storage solutions to support advanced analytics and machine learning initiatives.
Role type
Data Engineer
Builds
Data pipelines, ingestion frameworks, data warehouses, and lakes
Domain
Media and technology, cloud and on-prem data infrastructure
Deliverable
production ML models | infrastructure
Required skills
ETL process design, data pipeline development, data quality assurance, data governance, Kubernetes management, API development, data migration, cloud storage management, on-prem storage management
Preferred skills
Data lineage analysis, impact analysis, code review
Technologies
Kubernetes, Teradata, Databricks, minIO, AWS S3, Redshift
Responsibilities
Develop data structures and pipelines to collect, standardize, and transform data; Ensure data quality throughout ingestion, processing, and loading; Create ingestion frameworks for various data types; Maintain data warehouses and lakes; Determine suitable storage solutions based on data sensitivity; Handle data migrations and conversions; Design and monitor ETL processes; Participate in code reviews and data governance efforts.
Seniority
Mid-level (2-5 years experience)
