AI Inference Framework Engineer
Core
Integrate the GPNPU backend into AI inference frameworks (llama.cpp, PyTorch, vLLM) to enable running neural networks and C++ code on a single programmable architecture for edge devices.
Role type
Senior IC AI Inference Framework Engineer
Builds
AI inference backends for frameworks like llama.cpp, PyTorch, and vLLM
Domain
Edge AI, Neural Processing Units (NPU), Embedded Systems
Deliverable
production ML models
Required skills
C/C++, Python, Tensor memory management, Graph scheduling, Operator dispatch, AI compute performance optimization, AI coding agents (Claude Code, Cursor)
Preferred skills
Accelerator backend development (CUDA, Metal, Vulkan, Hexagon), Open-source framework contributions, Low-level AI compute pipelines, Graph compilers
Technologies
llama.cpp, PyTorch, vLLM, Optimum, GGUF, GPTQ, AWQ
Responsibilities
Implement framework backend interface, memory allocation, compute dispatch, and operators/kernels; Implement model compute pipelines including prefill, decode, batching, and multi-modal support; Optimize framework performance and accuracy for GPNPU backend; Validate accuracy against reference backends
Seniority
Senior, hands-on IC
