Development Engineer 2
Core
Design and implement scalable ETL/ELT processes, develop data models, and optimize Spark jobs for performance and cost efficiency.
Role type
Senior Data Engineer (Spark/Scala)
Builds
Scalable data pipelines, data models, and reliable datasets for business stakeholders
Domain
Media & Technology / Data Engineering
Deliverable
production ML models | infrastructure
Required skills
Apache Spark, Scala, ETL/ELT, distributed data processing, data modeling, performance optimization, data validation, DevOps collaboration (via careerplan.io/jobs/R437420-development-engineer-2-at-comcast)
Preferred skills
Streaming data processing, data lineage tracking, new technology research
Technologies
Apache Spark, Scala
Responsibilities
Design and implement scalable ETL/ELT processes using Apache Spark (batch and streaming); Optimize Spark jobs for performance and cost efficiency; Develop and maintain data models and schemas for structured and semi-structured data; Partner with DevOps teams for deployment and monitoring of Spark applications; Implement data validation, monitoring, and lineage tracking; Research and adopt new technologies to improve Spark-based data engineering processes.
Seniority
Mid-Senior, hands-on IC