Senior Software Engineer, ML API – AI Inference
Core
Building and evolving the ML API layer for a heterogeneous AI inference system combining GPU prefill with ultra-fast decode on Cerebras Wafer-Scale Engine.
Role type
Senior Software Engineer (ML API & Distributed Systems)
Builds
Production inference APIs, model integration layer, and request-routing services for AI model serving.
Domain
AI Infrastructure / Machine Learning Systems / Distributed Systems
Deliverable
production ML models
Required skills
Python, Go, C++, Rust, API design, distributed systems, Linux, Kubernetes, CI/CD, performance optimization, debugging, mentoring
Preferred skills
ML inference APIs, vLLM, PyTorch, Hugging Face, tokenization, disaggregated prefill/decode, Go/C++/Rust high-concurrency services, GPU inference (CUDA/ROCm), quantization
Technologies
vLLM, PyTorch, Hugging Face, LLGuidance, Cerebras runtime, Kubernetes, Linux, gRPC, REST
Responsibilities
Design, implement, test, and operate features for chat completions, streaming, and model configuration; Lead multi-component delivery and coordinate with partner teams; Implement request/response semantics across inference backends; Integrate foundation models and frameworks; Support disaggregated inference routing and lifecycle management; Diagnose and improve performance bottlenecks; Build conformance tests and validation suites; Strengthen production reliability and incident response; Improve developer experience and documentation; Mentor engineers and conduct code reviews.
Seniority
Senior, hands-on IC with leadership opportunities
