Software Engineer
Core
Build and operate a foundational model service platform serving LLMs and embedding models to Cisco IT engineering teams via Kubernetes clusters.
Role type
Senior IC software engineer (AI infrastructure & inference serving)
Builds
High-availability model serving platform on Kubernetes with Nim and vLLm runtimes
Domain
Enterprise IT infrastructure, Generative AI, Large Language Models
Deliverable
production ML models
Required skills
Kubernetes, Linux, GPU-based AI infrastructure, vLLM, NVIDIA NIM, Triton, model onboarding, deployment automation, monitoring, debugging
Preferred skills
Model benchmarking and evaluation, NVIDIA CUDA/AMD ROCm, distributed inference, KV cache aware routing, capacity planning, fine-tuning transformer models, RAG and agent systems
Technologies
Kubernetes, Nim, vLLm, NVIDIA NIM, Triton, Linux, CUDA, ROCm
Responsibilities
Support model onboarding and releases, maintain on-call rotation for critical services, automate validation gates for deployments, contribute to monitoring and observability, apply agentic solutions for operations, debug platform issues in dev and production
Seniority
Senior, hands-on IC