Cloud Data Engineer
Core
Design, build, and optimize scalable data pipelines across cloud and distributed systems for analytics and reporting.
Role type
Senior Cloud Data Engineer
Builds
Scalable data pipelines, data products, and metadata-driven frameworks
Domain
Cloud Data Engineering, BFSI (Banking)
Deliverable
production ML models | product features
Required skills
PySpark, SQL, Azure Data Lake Storage, AWS, Git, CI/CD, Control-M, API Testing, Root Cause Analysis
Preferred skills
Databricks Certified Data Engineer Associate
Technologies
Azure, AWS, Hadoop, Hive, Spark SQL, JIRA, Confluence, Bitbucket, Postman, PyCharm, VS Code, Control-M, Sterling B2B Integrator
Responsibilities
Build and maintain scalable data pipelines using cloud services; Develop data products integrating customer data; Create metadata-driven frameworks for data quality; Write PySpark and Spark SQL scripts for complex transformations; Support ETL processes by fixing errors and ensuring data integrity; Monitor data platforms and optimize pipelines for efficiency; Perform root cause analysis for job failures; Collaborate with onsite and offshore teams; Participate in design reviews and production deployments; Apply governance best practices for security and compliance; Lead projects through planning and coordination.
Seniority
Senior, hands-on IC