Staff Software Engineer, AI Inference Runtime
Core
Define architecture and roadmap for distributed AI inference runtime, optimizing scheduling, batching, memory management, and kernel performance for SOTA models.
Role type
Staff Software Engineer (AI Inference Runtime)
Builds
Distributed inference runtime components, optimized kernels, and production validation systems for Arm's AI platform.
Domain
AI Infrastructure / High-Performance Computing
Deliverable
production ML models
Required skills
C++, Rust, Python, concurrency, parallel programming, memory management, kernel development, profiling, debugging, optimization, distributed systems, model execution, KV-cache management, attention mechanisms, MoE, batching
Preferred skills
Inference schedulers, cache managers, disaggregated execution, accelerator programming, assembly, intrinsics, model parallelism, collective communication, high-performance networking, compilers, graph optimization, open-source contributions
Technologies
C++, Rust, Python, Arm hardware, distributed systems frameworks (via careerplan.io/jobs/100399715376-staff-software-engineer-ai-inference-runtime-at-arm)
Responsibilities
Define architecture and roadmap for AI inference runtime; Enable new model architectures via operator support and optimization; Profile system bottlenecks and develop optimized kernels; Evaluate inference techniques and build benchmarking/validation systems; Partner with cross-functional teams and mentor engineers.
Seniority
Staff, hands-on IC with strategic direction
