Data Engineer Analyst
Core
Design, build, and maintain scalable data pipelines and ingestion frameworks supporting batch processing, streaming, and change data capture (CDC) patterns.
Role type
Data Engineer
Builds
Scalable data pipelines, ingestion frameworks, and data lake architectures on AWS
Domain
Cloud Data Engineering (AWS)
Required skills
Apache Spark, PySpark, AWS Glue, AWS Athena, AWS Lambda, AWS S3, Apache Iceberg, AWS IAM, ETL orchestration, data quality monitoring
Preferred skills
CDC patterns, schema evolution, time-travel capabilities, data lake architecture design
Technologies
Apache Spark, PySpark, AWS Glue, AWS Athena, AWS Lambda, AWS S3, Apache Iceberg, AWS IAM
Responsibilities
Design and build scalable data pipelines; Develop and optimize data processing workflows; Implement and manage data lake architectures; Manage data access controls and security configurations; Collaborate with data analysts, data scientists, and platform engineers; Monitor pipeline health and troubleshoot data quality issues; Contribute to documentation of data flows and engineering standards
