Pyspark Data Engineer
Core
Design, develop, and maintain scalable data pipelines for large-scale datasets using PySpark and Cloudera Data Platform.
Role type
Senior Data Engineer (Big Data)
Builds
Scalable data ingestion and transformation processes
Domain
Big Data, Cloud-native data processing
Deliverable
production ML models | product features
Required skills
PySpark, Cloudera Data Platform (CDP), Hadoop, Hive, HDFS, ETL processes, data warehousing, distributed data processing
Preferred skills
Cloud environments, big data architecture, performance optimization
Technologies
PySpark, CDP, Hadoop, Hive, HDFS, Spark
Responsibilities
Design and develop scalable data pipelines, build and optimize data ingestion/transformation, ensure data quality and integrity, monitor and troubleshoot data workflows
Seniority
Senior, hands-on IC
Sourced via workday · Listed on CareerPlan, which tracks 70,000+ jobs from 20+ sources.
