Data Engineer (Spark)
Core
Design and maintain high-performance data processing platforms for automotive, aerospace, and telecommunications clients using streaming and batch processing.
Role type
Senior Data Engineer (Big Data)
Builds
Scalable data pipelines, data lakes, and centralized reporting platforms for enterprise clients.
Domain
Big Data, Cloud Infrastructure, Automotive, Aerospace, Telecommunications
Required skills
Python, Apache Spark, Cloudera, Airflow, Data Modeling, Cloud Infrastructure, Data Lake Management, Streaming/Batch Processing, OOP Design
Preferred skills
MLOps frameworks (Kubeflow, MLFlow), Databricks, dbt
Technologies
Spark, Cloudera, Airflow, Iceberg, Python, AWS, Kafka, Docker, BigQuery, Looker, Azure
Responsibilities
Develop and maintain high-performance data processing platforms; Design and implement data pipelines for streaming and batch modes; Optimize data workflows for ingestion, processing, and storage; Manage data lake technologies; Collaborate with cross-functional teams on data requirements; Monitor and troubleshoot platform availability and performance; Leverage cloud services for infrastructure scaling; Write and maintain high-quality code for automation.
Seniority
Senior, hands-on IC
