Software Engineer, Model Routing & Inference
Core
Build the inference platform powering every AI interaction, managing the full inference path for speed, reliability, and cost-efficiency at massive scale.
Role type
Senior IC software engineer (inference platform & routing)
Builds
Inference gateway, cross-provider failover systems, routing backpressure and admission control
Domain
AI infrastructure, distributed systems, high-throughput low-latency systems
Deliverable
production ML models
Required skills
high-throughput distributed systems, inference serving, traffic routing, real-time data pipelines, cost/performance tradeoffs at scale, production system architecture
Preferred skills
GPU utilization optimization, provider economics, capacity planning
Technologies
inference serving frameworks, traffic routing systems, distributed system components
Responsibilities
Build and evolve inference gateway abstraction over provider APIs, design intelligent cross-provider failover mechanisms, design routing backpressure and admission control for traffic spikes
Seniority
Senior, hands-on IC
