ML Engineer
Core
Building and scaling custom inference engines and low-latency serving runtimes for on-device AI, optimizing model intelligence and inference efficiency across diverse hardware architectures.
Role type
Senior IC ML Engineer (Systems & Inference)
Builds
Custom inference engines, cross-platform silicon optimization kernels, and robust serving runtimes for on-device AI models.
Domain
AI Inference, Edge Computing, Systems Programming
Deliverable
production ML models
Required skills
ML engineering, systems programming (Rust, C/C++), GPU programming, hardware optimization, LLM architecture understanding, performance profiling, quantization, speculative decoding
Preferred skills
ML compilers (torch.compile), low-precision inference (INT8/FP8/FP4), Edge LLMOps
Technologies
CUDA, ROCm, Metal, Triton, C++, Apple Silicon, NVIDIA, AMD, Snapdragon
Responsibilities
Develop custom inference engines handling weight loading, KV-cache management, and request scheduling; Write and tune custom kernels for diverse hardware architectures; Develop low-latency serving runtimes in C++ for model routing and decoding strategies; Identify and eliminate performance bottlenecks across the inference stack.
Seniority
Senior, hands-on IC