Senior Data Engineer - Spark & AWS
Core
Designing, building, and supporting scalable data pipelines that ingest telemetry, log, and configuration data for mission-critical business initiatives.
Role type
Senior hands-on IC data engineer (Spark & AWS)
Builds
Scalable ETL/ELT workflows and distributed data processing applications within a cloud-based data lake.
Domain
Telecommunications / Big Data / Cloud Infrastructure
Deliverable
production ML models | product features | infrastructure
Required skills
Apache Spark, Python, AWS cloud services, distributed systems, ETL/ELT pipeline design, large-scale data processing
Preferred skills
Scala, Java, streaming data technologies, graph-based data models, AI/ML data preparation, Agile methodologies
Technologies
AWS, Apache Spark, Python, Scala, Java
Responsibilities
Design and build scalable data pipelines ingesting telemetry, log, and configuration data; Develop distributed data processing applications using Apache Spark; Create and refine ETL/ELT workflows within a cloud-based data lake; Partner with Data Scientists and AI Engineers to prepare data for analytics and machine learning use cases; Improve pipeline performance, scalability, and reliability; Monitor data quality, validate outputs, and troubleshoot issues; Work with Platform Engineering to deploy and maintain data processing applications in AWS; Contribute to data architecture decisions and engineering standards.
