Sr. Software Engineer, Training/Inference (C++)
Core
Design and optimize large-scale model serving systems for high-performance inference of Grok models to millions of users.
Role type
Senior IC software engineer (training/inference systems)
Builds
Distributed inference infrastructure, GPU kernels, and model serving platforms
Domain
AI/ML infrastructure, high-performance computing, GPU systems
Deliverable
production ML models
Required skills
C++ system programming, distributed systems architecture, GPU kernel optimization, low-latency system design, CI/CD infrastructure, benchmarking and profiling
Preferred skills
Large-scale high-concurrency production serving, GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM), system optimizations (batching, caching, load balancing), algorithmic inference optimizations (quantization, speculative decoding, distillation)
Technologies
C++, C, Rust, vLLM, SGLang, Triton, TensorRT-LLM, GPU kernels
Responsibilities
Architect scalable distributed infrastructure for model serving; Optimize latency and throughput of model inference; Build high-concurrency serving systems; Benchmark and accelerate inference engines; Develop custom tools for tracing and debugging; Create robust CI/CD infrastructure; Accelerate research on scaling test-time compute and model-hardware co-design
Seniority
Senior, hands-on IC
