Staff Software Engineer, Foundational Model Serving

## Responsibilities - Design and implement core systems and APIs that power Databricks Foundation Model Serving, ensuring scalability, reliability, and operational excellence. - Partner with product and engineering leadership to define the technical roadmap and long-term architecture for serving workloads. - Drive architectural decisions and trade-offs to optimize performance, throughput, autoscaling, and operational efficiency for GPU serving workloads. - Contribute directly to key components across the serving infrastructure — from working in systems like vLLM and SGLang to creating token based rate limiters and optimizers — ensuring smooth and efficient operations at scale. - Collaborate cross-functionally with product, platform, and research teams to translate customer needs into reliable and performant systems. - Establish best practices for code quality, testing, and operational readiness, and mentor other engineers through design reviews and technical guidance. - Represent the team in cross-organizational technical discussions and influence Databricks’ broader AI platform strategy. ## Requirements - 10+ years of experience building and operating large-scale distributed systems. - Experience leading high-scale operationally sensitive backend systems. - A track record of up-leveling teams engineering excellence. - Strong foundation in algorithms, data structures, and system design as applied to large-scale, low-latency serving systems. - Proven ability to deliver technically complex, high-impact initiatives that create measurable customer or business value. - Strong communication skills and ability to collaborate across teams in fast-moving environments. - Strategic and product-oriented mindset with the ability to align technical execution with long-term vision. - Passion for mentoring, growing engineers, and fostering technical excellence. ## Nice to Have - Prior ML or AI experience. ## Benefits - Competitive salary and benefits package. - Opportunities to work on cutting-edge AI infrastructure. - Collaborative and innovative work environment.