Senior Software Engineer - Distributed Data Systems
Core
Building next-generation distributed data storage and processing systems (Apache Spark, Delta Lake, Delta Pipelines) to outperform SQL engines and support diverse workloads from ETL to data science.
Role type
Senior IC distributed systems software engineer (via careerplan.io/jobs/6936994002-senior-software-engineer-distributed-data-systems-at-databricks)
Builds
Distributed data storage, query optimizers, execution engines, and orchestration tools for the Databricks Data Intelligence Platform
Domain
Big data infrastructure, cloud storage, distributed systems
Deliverable
production ML models | product features
Required skills
Java, Scala, C++, algorithms and data structures, distributed systems, databases, big data systems (Apache Spark, Hadoop)
Preferred skills
multi-year vision planning, customer impact focus
Technologies
Apache Spark, Delta Lake, Delta Pipelines, AWS S3, Azure Blob Store
Responsibilities
Develop Apache Spark framework, build data plane storage services, implement Delta Lake storage management, orchestrate Delta Pipelines, build query optimizers and execution engines
Seniority
Senior, hands-on IC
