Staff Software Engineer - Ingestion
Core
Building and operating high-scale data ingestion systems (Lakeflow Connect) to extract data from OLTP systems and various sources with minimal production load, serving Databricks' data and AI surfaces.
Role type
Staff Software Engineer (Ingestion Systems)
Builds
Scalable, fault-tolerant data ingestion engine processing hundreds of TB daily
Domain
Data Infrastructure / Distributed Systems / Database Internals
Deliverable
production ML models | infrastructure
Required skills
Database internals, Database replication, Transaction recovery, Log parsing, Incremental data capture, Large-scale distributed systems, Algorithms and data structures, Low-level systems debugging, Performance optimization
Preferred skills
Multi-year vision planning, Customer value delivery, Technical leadership, Mentoring
Technologies
SQL Server, Oracle, IBM DB2, Apache Spark, Delta Lake
Responsibilities
Design architecture and influence product roadmap for ingestion capabilities, Debug and optimize large production clusters, Lead complex technical projects across multiple teams, Mentor team members and lead sprint planning, Investigate and resolve production issues, De-risk complex problems through prototyping
Seniority
Staff, hands-on IC with strategic influence
