ML Software Engineer, Data Plane
Core
Design and implement the inference data plane for large language models running on custom hardware, covering model execution, memory management, and data movement.
Role type
Senior IC software engineer (ML inference stack)
Builds
High-performance compute kernels, custom accelerator backends, and serving integration for large distributed models
Domain
AI Infrastructure / Custom Hardware Acceleration
Deliverable
production ML models
Required skills
C/C++, Linux systems, computer architecture, parallel computing, compute kernel development, end-to-end feature ownership
Preferred skills
ML frameworks (PyTorch, vLLM, JAX), LLM fundamentals, distributed systems, AI-assisted development tools
Technologies
PyTorch, vLLM, C++, Linux, RDMA
Responsibilities
Develop and optimize compute kernels for custom ML accelerators; Integrate custom backends into open-source serving frameworks; Build test infrastructure for model correctness; Profile and optimize inference workloads; Own features from design through production integration; Contribute to CI/CD pipelines for performance gating