Member of Technical Staff — Model Optimization and Inference (New Grad)
Core
Optimizing real-time inference for full-duplex multimodal AI avatars to achieve sub-500ms latency.
Role type
New Grad Model Optimization and Inference Engineer
Builds
Real-time conversational AI avatars with emotional intelligence
Domain
AI/ML Systems, Real-time Inference, Multimodal Systems
Deliverable
production ML models
Required skills
LLM inference optimization, KV cache management, inference serving frameworks, Python, PyTorch, profiling and benchmarking, quantization techniques, CUDA/Triton familiarity
Preferred skills
Diffusion model inference acceleration, speculative decoding, open-source contributions to inference frameworks, hard latency SLA experience
Technologies
vLLM, SGLang, TensorRT-LLM, CUDA, Triton, PyTorch
Responsibilities
Implement and tune KV cache strategies for long-context conversations, Profile and benchmark end-to-end latency and throughput, Build internal tooling for optimization work, Accelerate diffusion model inference, Apply quantization techniques to reduce memory footprint
Seniority
Junior, New Grad
