Senior Software Developer: Models Team (Token Factory)
Core
Building and optimizing production-scale infrastructure for serving large language models (LLMs) efficiently and reliably at scale.
Role type
Senior Software Developer (LLM Inference Systems)
Builds
High-performance AI inference platform (Token Factory) serving open-source models like DeepSeek, GLM, and Kimi.
Domain
Cloud Infrastructure / AI Systems / Distributed Computing
Deliverable
production ML models
Required skills
LLM production serving, Python, Go, distributed systems design, high availability
Preferred skills
KV cache management, speculative decoding, quantization, LLM evaluation, Kubernetes, InfiniBand/RoCE networking
Technologies
vLLM, TRT-LLM, Go, Python, Kubernetes, InfiniBand, RoCE
Responsibilities
Onboard state-of-the-art open-source models into the platform, implement advanced inference optimization techniques (cache-aware routing, NUMA-aware deployments, KV-cache offloading), maintain and extend inference frameworks, build tooling for performance testing and automated rollouts.
Seniority
Senior, hands-on IC