AI Infrastructure Engineer - Generative AI Model Serving (Go / Rust / GPU)
Core
Build the serving layer for large language models and generative AI within the Teradata Vantage ecosystem, focusing on GPU-accelerated inference, high-performance services, and cloud-native platform delivery.
Role type
Senior IC AI Infrastructure Engineer (Generative AI Model Serving)
Builds
Scalable, low-latency model-serving platforms, GPU cluster infrastructure, and developer tooling for AI/ML teams.
Domain
Enterprise AI / Generative AI / Distributed Systems
Deliverable
production ML models
Required skills
Go, Rust, GPU computing (CUDA), LLM inference optimization, Kubernetes, CI/CD, System design, Observability, Full-stack engineering
Preferred skills
CUDA kernel tuning, On-device ML inference, RAG pipelines, Service meshes
Technologies
Go, Rust, Kubernetes, vLLM, TensorRT-LLM, Triton Inference Server, Ray Serve, KServe, SGLang, llama.cpp, ONNX Runtime, OpenTelemetry, Prometheus, Grafana, Terraform, React, PyTorch, Hugging Face
Responsibilities
Design and operate scalable model-serving platforms; Develop high-performance control-plane and data-plane services; Optimize GPU inference for throughput and latency; Integrate and extend inference engines; Build and manage GPU cluster infrastructure; Own the full delivery path including CI/CD and rollouts; Build AI observability solutions; Provide clean APIs and developer tooling; Support agentic and RAG workloads; Drive reliability, security, and cost efficiency.
Seniority
Senior, hands-on IC
