Staff AI Platform Engineer, Infrastructure Services
Core
Architect, harden, and scale the AI Gateway infrastructure (Kong AI Gateway) and self-hosted LLM serving stacks to secure and deliver AI capabilities for global enterprises.
Role type
Staff AI Platform Engineer (Infrastructure)
Builds
AI Gateway platform, self-hosted model serving infrastructure, and LLMOps tooling
Domain
Cybersecurity, AI/ML Infrastructure, Cloud Native
Deliverable
production ML models
Required skills
Kubernetes, API Gateway (Kong), CI/CD (Jenkins, GitHub Actions), GitOps (ArgoCD), Terraform, AWS/EKS, GPU infrastructure management, LLMOps practices
Preferred skills
LLM gateway patterns, self-hosted inference stacks (vLLM, Triton), vector databases, model fine-tuning, enterprise auth (Okta/OIDC)
Technologies
Kong, KCP, EKS, Okta, OIDC, Jenkins, JPAAS, ArgoCD, Artifactory, Xray, GitHub Enterprise, vLLM, NVIDIA Triton, TGI, Ollama, Milvus, Pinecone, pgvector
Responsibilities
Architect and scale the Kong AI Gateway deployment; lead reliability and incident response for gateway issues; design cross-platform solutions integrating CI/CD and artifact management; evaluate and roll out AI developer tooling; set technical direction and mentor engineers; host and serve local models with GPU capacity planning; support the broader model lifecycle including LLMOps and RAG pipelines; track usage and cost across AI infrastructure.
Seniority
Staff, high-autonomy technical leadership