Senior System Software Engineer - LocalAI
Core
Develop efficient on-device AI software for RTX and DGX-class systems to deliver high-performance local inference with low latency and optimized memory utilization.
Role type
Senior Systems Software Engineer (Local AI Inference)
Builds
Local AI inference stack, modern inference runtimes, and execution stacks for LLM, vision-language, TTS, ASR, and diffusion-based workloads.
Domain
AI Computing / GPU Systems / Edge Computing
Deliverable
production ML models
Required skills
C++ programming, CUDA, data structures, algorithms, machine learning, model optimization (quantization, pruning, sparsity, distillation), system-level debugging, performance tuning, memory management, graph execution, hardware-aware optimization
Preferred skills
Open-source contributions, low-level system programming, distributed team experience, high-performance system development
Technologies
llama.cpp, vLLM, PyTorch, WinML, DXCGC, TensorRT-RTX, Vulkan, DirectX
Responsibilities
Partner with software, research, architecture, and product teams to align technical requirements; Build and optimize local AI inference stack for diverse hardware architectures; Design and develop inference runtimes and execution stacks; Perform end-to-end optimization of AI models and data pipelines; Conduct system-level debugging, performance tuning, and accuracy trade-off analysis.
Seniority
Senior, hands-on IC
