Software Engineer AI Gateway
Core
Build and maintain API management software for AI and LLMs, including multi-LLM routing, balancing, semantic caching, and proxying capabilities.
Role type
Senior IC software engineer (AI gateway infrastructure)
Builds
Kong AI Gateway (multi-LLM routing, balancing, semantic caching, MCP, proxying, load balancing, database support)
Domain
Developer tools / API infrastructure / AI & LLMs
Deliverable
production ML models | product features
Required skills
Golang or Rust, distributed systems, Linux systems programming, Linux networking stack (L7/L4), low-level optimization, scalable application development
Preferred skills
NGINX/OpenResty/Kong/LuaJIT, Envoy, Istio, AI/LLM concepts
Technologies
Golang, Rust, Linux, Kong, NGINX, OpenResty, LuaJIT, Envoy, Istio
Responsibilities
Build and maintain API management software for AI/LLMs; Implement innovative features like proxying and load balancing; Troubleshoot bugs and tune performance; Bridge gaps between Kong and third-party platforms
Seniority
Mid-Senior, hands-on IC
