Senior AI Serving Engineer, Backend
Core
Build the model serving platform (API, Control Plane, Billing, Monitoring, distributed inference) to power real-time multimodal AI applications.
Role type
Senior IC backend engineer (AI serving infrastructure)
Builds
Production model serving platform and distributed inference features
Domain
AI infrastructure / High-performance computing
Deliverable
production ML models
Required skills
C++, Python, Go, Rust, Kubernetes, Containerization, Large-scale ML/MLOps infrastructure, Performance engineering, Runtime execution, API development, Distributed systems
Preferred skills
CUDA, ROCm, GPU profiling tools, Open-source inference engines (vLLM, Sglang, TRT-LLM), HPC infrastructure
Technologies
C++, Python, Go, Rust, Kubernetes, vLLM, Sglang, TRT-LLM, CUDA, ROCm
Responsibilities
Build API, Control Plane, Billing, and Monitoring components for the serving platform; Integrate new multimodal models into production workflows; Provide operational support for production services; Troubleshoot complex issues across runtime, service, and GPU layers
Seniority
Senior, hands-on IC
