Sr. Databricks Data Engineer, Onsite AVP
Core
Design, build, and optimize high-throughput data pipelines and end-to-end data ingestion solutions (streaming, events, batch) on Databricks for State Street Global Technology Services.
Role type
Senior Databricks Data Engineer
Builds
Scalable ETL pipelines, Data Lakehouse solutions, and cloud-based data infrastructure
Domain
Financial Services / Big Data Engineering
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Databricks SQL, Scala/PySpark, Azure services, Microservices/API development, ETL pipeline design, Query optimization, Resource management, Root cause analysis, Software development lifecycle management
Preferred skills
Financial industry experience, Agile methodology, Cross-functional collaboration
Technologies
Databricks, Azure, Scala, PySpark, SQL
Responsibilities
Design and develop custom high-throughput data pipelines; Build and test end-to-end data ingestion pipelines; Optimize Databricks job performance and scaling strategies; Address data and environment issues through impact and root cause analysis; Manage all aspects of the software development lifecycle
Seniority
Senior, hands-on IC
