PySpark Developer
Core
Design, build, validate, and support data solutions for enterprise operations, analytics, and decision-making, focusing on data integration, modeling, quality, performance, and governance.
Role type
Senior PySpark Data Engineer
Builds
Data pipelines, transformations, reports, models, and platform components
Domain
Data Engineering / Big Data Processing
Deliverable
production ML models
Required skills
Apache Spark, PySpark, Scala, Spark SQL, distributed processing, data modeling, ETL/ELT, data quality, SQL, performance tuning
Preferred skills
Cloud services, automation, modern delivery practices, secure development, governance
Technologies
Apache Spark, PySpark, Scala, Spark SQL
Responsibilities
Develop and maintain data pipelines, transformations, reports, models, or platform components; Translate source-to-target and business requirements into validated technical data solutions; Implement data quality checks, access controls, monitoring, and failure-recovery procedures; Tune SQL, processing jobs, models, dashboards, or platform resources for performance and cost; Document data definitions, lineage, operational procedures, and technical decisions
Seniority
Senior, hands-on IC