Software Engineer, Model API Infra
Core
Design, implement, and operate scalable API services and agentic platforms for text, multimodal, and emerging MAI models, ensuring training and production parity and automating the model deployment lifecycle.
Role type
Senior IC software engineer (model API infrastructure)
Builds
Production APIs, agentic platforms, and model serving automation systems
Domain
AI/ML infrastructure, cloud services, distributed systems
Deliverable
production ML models
Required skills
Python, C++, distributed systems design, Kubernetes, cloud platforms, systems debugging, deployment automation, autoscaling
Preferred skills
LLM serving concepts (streaming inference, token-based capacity, batching, KV cache), agent runtimes, post-training workflows, open-source ML frameworks
Technologies
Kubernetes, Azure Machine Learning, Microsoft Foundry
Responsibilities
Design and operate scalable API services for multimodal models; build closed-loop systems for automated model deployment and validation; design autoscaling for model pools using latency SLOs; build telemetry for cross-system debugging; partner with product and research teams to deliver model capabilities.
Seniority
Senior, hands-on IC