Software Engineer, Training/Inference (C++)
Core
Architect and implement scalable distributed model-serving infrastructure and optimize low-level inference performance for large-scale LLMs.
Role type
Senior IC software engineer (LLM inference systems)
Builds
High-performance, low-latency inference serving systems for millions of users
Domain
Artificial Intelligence / Large Language Models / Systems Engineering
Deliverable
production ML models
Required skills
C++, Rust, GPU inference engines, systems optimization, distributed systems, CI/CD, low-level inference optimization
Preferred skills
GPU kernel development, speculative decoding, quantization, test-time compute scaling
Technologies
vLLM, SGLang, Triton, TensorRT-LLM, C++, Rust, CI/CD, Load Balancing, Model Serving
Responsibilities
Architect scalable distributed model-serving infrastructure with global KV cache; Optimize inference latency and throughput under production workloads; Build and operate highly concurrent serving systems; Benchmark and tune inference engines via GPU kernel development; Develop tools for tracing and resolving full-stack issues; Create robust CI/CD infrastructure for endpoint deployment; Advance research into test-time compute scaling and model-hardware co-design.
