Big Data Engineer
Core
Build and maintain ETL pipelines, orchestrate workflows, and design storage/querying solutions for Fortune 500 clients in financial, healthcare, and manufacturing sectors.
Role type
Mid-level IC Big Data Engineer (AWS)
Builds
ETL pipelines, data lakes, and cloud-native data architectures for enterprise clients
Domain
Cloud data engineering / Big Data
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Python, PySpark, SQL, AWS Glue, AWS Step Functions, AWS Lambda, AWS SNS, AWS SQS, Amazon Redshift, Amazon S3, API integration, CI/CD, Terraform, GitLab, data warehousing, data lakes
Preferred skills
Athena, EMR, Kinesis, DynamoDB, RDS, CloudWatch, performance engineering, AI coding assistants, enterprise data lake migration
Technologies
AWS (Glue, Step Functions, Lambda, Redshift, S3, SNS, SQS, Athena, EMR, Kinesis, DynamoDB, RDS), Python, PySpark, SQL, Terraform, GitLab
Responsibilities
Build and maintain ETL pipelines using Python and PySpark on AWS Glue; Orchestrate workflows using AWS Step Functions and Lambda; Implement messaging and event-driven integrations using SNS and SQS; Design and optimize storage and querying solutions in Amazon Redshift, RDS, Oracle and S3-based architectures; Write efficient SQL for transformations, validation, and reporting; Integrate data from APIs and process structured and semi-structured JSON data; Implement data quality checks, monitoring, and operational support processes; Participate in CI/CD and version control practices for deployment and release management
Seniority
Mid-level, hands-on IC
