Sr. Software Engineer, Training/Inference
Core
Architect and implement distributed model-serving infrastructure to optimize inference latency, throughput, and reliability at production scale.
Role type
Senior IC systems engineer (LLM inference serving)
Builds
Distributed model-serving infrastructure, inference engines, and deployment pipelines
Domain
AI/ML infrastructure, GPU computing, systems programming
Deliverable
production ML models
Required skills
C/C++ or Rust, GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM), distributed systems, low-level systems programming, GPU kernels, code generation, quantization, speculative decoding, CI/CD
Preferred skills
Research on test-time compute scaling, RL rollout, model-hardware co-design
Technologies
vLLM, SGLang, Triton, TensorRT-LLM, C++, Rust, GPU
Responsibilities
Architect distributed model-serving infrastructure; Optimize inference latency and throughput; Benchmark and accelerate inference engines; Develop tools for tracing and issue resolution; Build CI/CD infrastructure for endpoint deployment; Advance research on inference scaling and co-design
Seniority
Senior, hands-on IC
