Data Engineer II
Core
Constructing and optimizing high-performance data pipelines for batch and streaming data processing, leading Proof-of-Concept initiatives for workflow orchestration and Generative AI integration.
Role type
Mid-level IC data engineer (cloud & big data)
Builds
Scalable data pipelines, data processing jobs, and workflow orchestration tools on AWS
Domain
Telecommunications / Big Data Engineering
Deliverable
production ML models | product features | infrastructure
Required skills
Python, PySpark, SQL, AWS EMR, Apache Airflow, GitLab CI/CD, Terraform, Generative AI tools
Preferred skills
Databricks
Responsibilities
Design and optimize scalable data pipelines for batch and streaming; Develop and manage data processing jobs on Amazon EMR; Implement transformation logic using Python, PySpark, and SQL; Lead POC for Apache Airflow workflow orchestration; Integrate Amazon Q for automated data quality and documentation; Implement CI/CD pipelines using GitLab and Infrastructure-as-Code.
Seniority
Mid-level, hands-on IC
