Senior Data Engineer (Platform)
Core
Design, build, and maintain scalable batch and streaming data pipelines on AWS to consolidate data from multiple systems into trusted analytical datasets for reporting and business intelligence.
Role type
Senior IC data engineer (cloud data platform)
Builds
Production data pipelines, data lake solutions, and analytical data models
Domain
Cloud data engineering, data warehousing, and analytics
Deliverable
production ML models | product features | dashboards & analysis
Required skills
AWS data services (Redshift, Glue, S3, Kinesis), Python, SQL, Apache Spark, data modeling (star/snowflake schemas), ETL/ELT pipeline design, workflow orchestration (Airflow, Step Functions), distributed processing, data governance, Infrastructure as Code, CI/CD
Preferred skills
Apache Iceberg/Delta Lake/Hudi, dbt, Kafka/MSK, Lakehouse architectures, Terraform, containerized workloads (Docker/ECS/EKS), DataOps, BI platforms (Tableau/Power BI/Looker)
Technologies
Python, SQL, PySpark, Apache Spark, AWS Glue, Databricks, Amazon S3, Amazon Redshift, Parquet, Amazon Kinesis Firehose, EventBridge, Apache Airflow (MWAA), AWS Step Functions, Terraform, CloudFormation, CloudWatch, Datadog, Git, GitHub Actions
Responsibilities
Design and implement distributed data processing jobs; optimize Redshift performance; build resilient pipelines with retry and checkpointing; implement automated data quality validation and lineage tracking; develop infrastructure and deployment automation; monitor and troubleshoot production data infrastructure; collaborate with cross-functional teams to translate requirements into scalable solutions
Seniority
Senior, hands-on IC
