Senior Software Engineer, AI Gateway
Core
Design and own core components of a highly performant, secure, and observable AI Gateway that connects enterprises to generative AI workloads, ensuring 99.99% uptime and low-latency performance for critical AI traffic.
Role type
Senior Software Engineer (Backend/Infrastructure)
Builds
AI Gateway infrastructure for integrating LLMs and AI services into enterprise applications at massive scale
Domain
Cloud-native infrastructure, API management, Generative AI
Deliverable
production ML models
Required skills
Golang, Python, distributed systems, microservices architecture, API design, Kubernetes, Docker, AWS/GCP/Azure, computer science fundamentals
Preferred skills
AI/ML infrastructure, LLMs, LangChain, LlamaIndex, open-source contributions, performance tuning, Lua/OpenResty
Technologies
Golang, Python, Kubernetes, Docker, AWS, GCP, Azure, LLMs, LangChain, LlamaIndex, Lua, OpenResty
Responsibilities
Design and own core components of the AI Gateway; Lead implementation of features like prompt engineering, caching, and observability; Influence technical roadmap and architecture; Architect robust solutions for integrating with various LLMs and AI platforms; Mentor junior engineers; Drive the full development lifecycle from ideation to deployment; Collaborate with product managers and cross-functional teams
Seniority
Senior, hands-on IC
