Lead Cloud Data Engineer (Databricks) - Assistant Vice President
Core
Design, build, and optimize scalable data pipelines and workflows on cloud platforms using Databricks and Spark to deliver high-quality, reliable data solutions.
Role type
Senior IC Lead Cloud Data Engineer (Databricks)
Builds
Scalable data pipelines, ETL/ELT workflows, and production-grade data platforms
Domain
Financial services / Big Data / Cloud Data Engineering
Deliverable
production ML models | product features
Required skills
Databricks, Apache Spark (Python/Scala), Cloud platforms (AWS/Azure), SQL, Python/Scala programming, Data pipeline design, ETL/ELT frameworks, Data modeling, Performance optimization, Scalability
Preferred skills
Streaming technologies (Kafka), Delta Lake, Data lakehouse architecture, CI/CD, DevOps, Containerization (Docker/Kubernetes), Data governance
Technologies
Databricks, Apache Spark, Kafka, Delta Lake, AWS, Azure, Docker, Kubernetes
Responsibilities
Design, develop, and maintain scalable data pipelines and workflows; Build and optimize ETL/ELT pipelines for batch and near real-time processing; Collaborate with architects and business stakeholders to translate requirements into technical implementation; Ensure data quality, governance, and consistency across data platforms; Perform performance tuning and optimize cost efficiency of data workloads; Support production deployments, troubleshoot issues, and ensure reliability of data systems
Seniority
Senior, hands-on IC
