Staff Backend Engineer - K8 (Envoy)
Core
Architect and build Coupang's next-generation AI Inference Gateway platform to serve mission-critical machine learning and generative AI workloads at scale.
Role type
Staff Backend Engineer (AI Infrastructure & Service Mesh)
Builds
Scalable, secure, and reliable AI inference infrastructure across cloud and on-prem environments.
Domain
E-commerce, Cloud Infrastructure, AI/ML Systems
Deliverable
production ML models | infrastructure
Required skills
Go, Java, Python, Kubernetes, AWS, distributed systems design, microservices, observability, reliability engineering, capacity planning, API gateway design, service mesh, traffic management, concurrency, non-blocking I/O
Preferred skills
AI/ML inference platforms, LLM gateways, GPU-accelerated workloads, Kubernetes ecosystem (Gateway API, Istio, Envoy), inference serving frameworks (vLLM, Triton, Ray Serve), high-performance networking (gRPC, HTTP/2), GenAI/LLM ecosystems, distributed data systems (Kafka, Redis)
Technologies
Go, Java, Python, Kubernetes, AWS, Kafka, Kubeflow, Argo CD, gRPC, Envoy, Istio, Linkerd, vLLM, Triton Inference Server, TensorRT-LLM, Ray Serve, KServe, SGLang
Responsibilities
Architect and build the AI Inference Gateway platform; Design high-performance request routing, load balancing, and policy enforcement; Drive technical vision for scalable AI infrastructure; Develop critical infrastructure components; Design multi-tenant platform capabilities; Partner with ML and Data teams; Lead architecture reviews and mentor senior engineers; Optimize system performance and latency; Define observability standards; Investigate production issues and implement architectural solutions.
Seniority
Staff, hands-on IC with strategic direction and mentorship