Big Data Engineer
Core
Design, develop, and maintain cloud-based data pipelines and ETL solutions on AWS.
Role type
Junior IC AWS data engineer
Builds
ETL pipelines and data workflows on AWS
Domain
Cloud data engineering
Deliverable
production ML models | product features
Required skills
Python, SQL, PySpark, AWS Glue, AWS Lambda, AWS Step Functions, AWS S3, AWS Redshift, AWS RDS, ETL concepts, GitLab, Terraform
Preferred skills
PySpark, AWS Athena, AWS CloudWatch, AWS SNS, AWS SQS, AI coding tools
Technologies
AWS, Python, PySpark, SQL, GitLab, Terraform
Responsibilities
Build and maintain ETL pipelines using Python and PySpark; Develop workflows using AWS Glue, Lambda, and Step Functions; Write SQL queries for data extraction, transformation, validation, and reporting; Implement basic monitoring, logging, and error handling for pipelines; Support ingestion and processing of data from APIs and JSON payloads; Contribute to code management, documentation, and deployment support activities
