Runtime Engineer
Core
Design and implement a runtime layer for loading, scheduling, and executing compiled programs on specialized AI hardware that performs memory-bound inference inside memory.
Role type
Senior IC Runtime Engineer (AI hardware)
Builds
A stable runtime interface for large-model inference on pre-silicon hardware (simulators, FPGA, future silicon)
Domain
Semiconductor / AI Infrastructure / Systems Programming
Deliverable
production ML models
Required skills
Systems programming in C++ or Rust, Runtime architecture design, Hardware abstraction, Memory management, Concurrency, Low-level performance optimization, LLM inference workload understanding, Cross-team interface definition
Preferred skills
Experience with IREE HAL, CUDA, or ROCm runtime internals, In-memory compute experience, Inference server integration (e.g., vLLM), Open-source contributions
Technologies
C++, Rust, FPGA, Simulators, LLM inference engines
Responsibilities
Own runtime architecture design including device management and memory allocation, Hide hardware differences behind a stable interface, Manage device state for serving layers, Optimize runtime overhead for end-to-end latency and throughput, Define boundaries with compiler and driver teams, Set technical direction and review dependent designs
Seniority
Senior, hands-on IC