Senior Data Engineer - PySpark & Python
Core
Design, develop, and maintain scalable ETL pipelines, data marts, and production-grade data engineering solutions for enterprise-scale Data & Analytics initiatives.
Role type
Senior IC data engineer (PySpark/Python)
Builds
Scalable ETL pipelines, data marts, and production-grade data engineering solutions
Domain
Financial services / Big Data
Deliverable
production ML models | product features | infrastructure
Required skills
Python (expert), PySpark (expert), ETL pipeline development, Data Mart development, Data Warehousing, End-to-End SDLC, Apache Spark, Hadoop, MapReduce, Hive, Pandas, SQL, NoSQL, Oracle SQL, Apache Airflow, Oozie, Jenkins Pipelines, Git, CI/CD, Testing & Validation, Production Deployment
Preferred skills
Banking domain experience, Digital Products experience, Enterprise-scale Data & Analytics platforms experience, Agile methodologies
Technologies
Apache Spark, Hadoop, MapReduce, Hive, Pandas, Oracle, Apache Airflow, Oozie, Jenkins, Git, Jupyter Notebook
Responsibilities
Design and maintain scalable ETL pipelines and data marts using PySpark and Python; Build robust, maintainable, and production-ready data engineering solutions; Perform end-to-end SDLC activities including development, UAT support, bug fixes, production deployments, and post-production support; Debug and optimize PySpark code and complex SQL queries for performance and scalability; Develop and maintain production-grade data pipelines using modern data engineering best practices; Ensure data quality, integrity, and consistency across enterprise data platforms; Participate in CI/CD implementation, testing, validation, and deployment activities
Seniority
Senior, hands-on IC
