Data Engineer – Python, PySpark, SQL & ETL
Core
Design, develop, optimize, and support scalable ETL pipelines and data marts using Python, PySpark, and SQL.
Role type
Senior Data Engineer (ETL & Data Warehousing)
Builds
Scalable data pipelines, data marts, and distributed data-processing solutions.
Domain
Enterprise data engineering and data warehousing
Required skills
Python, PySpark, Spark, SQL, Oracle SQL, ETL development, Data warehousing, Hadoop, Hive, MapReduce, CI/CD pipelines, Data validation
Preferred skills
Banking/financial-services data, Cloud-based data platforms, Workflow orchestration, Data quality automation, Infrastructure automation
Technologies
Python, PySpark, Spark, Pandas, Oracle SQL, Hadoop, Hive, MapReduce, Jupyter Notebook
Responsibilities
Design and develop scalable ETL pipelines; Build and enhance data marts; Develop distributed data-processing solutions; Optimize Oracle SQL queries; Perform end-to-end SDLC activities; Monitor production environments and troubleshoot pipeline failures; Implement data validation and reconciliation checks.
Seniority
Senior, hands-on IC