Senior Data Engineer
Core
Design and maintain robust data ingestion pipelines and frameworks for a data lake, ensuring high data reliability and seamless delivery to business teams.
Role type
Senior IC data engineer
Builds
Data ingestion processes, monitoring frameworks, and AWS-based data lake infrastructure
Domain
Media & Publishing / Cloud Data Engineering
Required skills
Python (Pandas, AWS Wrangler), SQL (DDL/DML), AWS services (Athena, Glue, S3, Lambda, ECS, Redshift), Infrastructure-as-Code, unit testing, pipeline observability
Preferred skills
Experience with diverse data sources (APIs, SFTP), ability to explain complex technical concepts to non-technical partners
Technologies
Python, AWS (Athena, Glue, S3, Lambda, ECS, Redshift), IaC, Pandas, AWS Wrangler, SQL
Responsibilities
Design and build data ingestion processes from diverse sources; Provision AWS infrastructure using IaC; Embed observability mechanisms for data reliability; Partner with analysts and product managers to ensure data delivery; Write documentation and enforce unit testing standards
Seniority
Senior, hands-on IC
