Python Pyspark Developer
Core
Design, develop, and maintain ETL/data pipelines for large-scale data processing and batch transformation.
Role type
Senior IC Python/PySpark Data Engineer
Builds
Scalable ETL pipelines and data transformation workflows
Domain
Banking / Big Data
Deliverable
production ML models | product features
Required skills
Python, PySpark, SQL, data modeling, performance tuning, batch processing, data quality validation, error handling, business requirement analysis
Preferred skills
AWS, Azure, Databricks
Technologies
PySpark, SQL, AWS, Azure, Databricks
Responsibilities
Design and develop ETL/data pipelines using PySpark and Python; Optimize data processing performance; Analyze business requirements and translate them into scalable technical solutions; Ensure data quality, reconciliation, and validation
Seniority
Senior, hands-on IC
Sourced via workable · Listed on CareerPlan, which tracks 70,000+ jobs from 20+ sources.
