Member of Technical Staff — Model Optimization and Inference (Experienced)
Core
Optimizing real-time inference for full-duplex multimodal AI avatars to achieve sub-500ms latency.
Role type
Senior IC model optimization and inference engineer
Builds
Production-grade, real-time audiovisual AI avatar systems
Domain
AI/ML, Real-time Systems, Multimodal
Deliverable
production ML models
Required skills
LLM inference optimization, KV cache strategies, inference serving frameworks (vLLM, SGLang, TensorRT-LLM), diffusion model inference acceleration, quantization techniques (INT8, INT4, GPTQ, AWQ), CUDA/Triton kernel development, Python, PyTorch
Preferred skills
Post-training quantization, speculative decoding, multimodal/streaming inference architectures, real-time system deployment with hard latency SLAs, open-source inference framework contributions
Technologies
vLLM, SGLang, TensorRT-LLM, CUDA, Triton, PyTorch
Responsibilities
Own end-to-end inference optimization across LLM, audio, and diffusion components; Implement and tune KV cache strategies; Evaluate and extend inference serving frameworks; Profile and benchmark latency/throughput; Build internal optimization tooling; Accelerate diffusion model inference via step distillation and custom kernels
Seniority
Senior, hands-on IC
