Senior Staff Software Engineer - AI/ML Inference Engineering
Core
Lead the architecture, optimization, and operational excellence of the AI inference platform, owning the end-to-end inference lifecycle from model onboarding to production deployment across Kubernetes backends.
Role type
Senior Staff Software Engineer (AI/ML Inference)
Builds
Optimized inference engines, intelligent gateway layer, and production AI services
Domain
Enterprise Software / AI Infrastructure / LLM Serving
Deliverable
production ML models
Required skills
Go or Python, OOP, Design Patterns, time and space-efficient algorithms, distributed systems, Kubernetes, Envoy, LLM APIs, inference frameworks (vLLM, TensorRT-LLM, Triton), unit testing, profiling, code tuning
Preferred skills
CI/CD pipelines, observability (metrics, logs, tracing), incident management, root-cause analysis, AI coding assistants (GitHub Copilot, Cursor), data privacy rules
Technologies
Kubernetes, Envoy, Helm, vLLM, TensorRT-LLM, Triton Inference Server, GitHub Copilot, Cursor
Responsibilities
Architect intelligent gateway layer using Envoy and ext_proc, deploy and scale services in production Kubernetes environments, debug and resolve production incidents independently, own observability for built services
Seniority
Senior Staff, hands-on IC
