Data Engineer (Python/PySpark)
Core
Design, develop, and maintain scalable ETL/Data Pipeline solutions and Data Marts for large-scale data processing.
Role type
Senior Data Engineer
Builds
ETL pipelines, Data Marts, data transformation processes
Domain
Banking / Financial Services
Deliverable
production ML models
Required skills
Python, PySpark, SQL, Data Warehousing, ETL development, Hadoop, Hive, MapReduce, NoSQL databases, CI/CD pipelines, data validation
Preferred skills
Banking domain experience
Technologies
Python, PySpark, Spark, Pandas, Oracle, Hadoop, Hive, MapReduce, SQL, NoSQL, Jupyter Notebook, CI/CD Tools
Responsibilities
Design and maintain scalable ETL/Data Pipeline solutions, Build and optimize Data Marts, Perform end-to-end SDLC activities, Analyze and debug data issues, Collaborate with cross-functional teams for data quality and testing, Support production environments and troubleshoot pipeline issues
Seniority
Senior, hands-on IC
