Big Data Engineer
Core
Design, build, and maintain ETL pipelines and data integration solutions on AWS to support enterprise data warehousing and data lake architectures.
Role type
Mid-level IC Big Data Engineer
Builds
ETL pipelines, event-driven integrations, and optimized storage/querying solutions for data warehousing and data lakes
Domain
Cloud Data Engineering (AWS)
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Python, PySpark, SQL, AWS Glue, AWS Step Functions, AWS Lambda, SNS, SQS, Amazon Redshift, S3, API integration, CI/CD, Terraform, GitLab, data quality monitoring, code optimization
Preferred skills
Athena, EMR, Kinesis, DynamoDB, RDS, CloudWatch, AI coding tools (e.g., GitHub Copilot), enterprise data lake migration, near real-time systems, Agents and MCP
Technologies
AWS Glue, AWS Step Functions, AWS Lambda, SNS, SQS, Amazon Redshift, S3, Athena, EMR, Kinesis, DynamoDB, RDS, GitLab, Terraform, Python, PySpark
Responsibilities
Build and maintain ETL pipelines using Python and PySpark; orchestrate workflows using AWS Step Functions and Lambda; implement messaging and event-driven integrations; design and optimize storage and querying solutions; write efficient SQL for transformations and reporting; integrate data from APIs and process JSON data; implement data quality checks and operational support; participate in CI/CD and version control practices.
Seniority
Mid-level, hands-on IC
