Senior Machine Learning Engineer, SageMaker Inference
Core
Design and implement kernel-level optimizations and inference pipeline improvements to reduce latency and increase throughput for LLM serving workloads on AWS SageMaker.
Role type
Senior IC machine learning engineer (inference optimization)
Builds
High-throughput, low-latency CPU/GPU/Inferentia inference workloads for Generative AI models
Domain
Cloud infrastructure + Generative AI
Deliverable
production ML models
Required skills
CUDA kernel optimization, LLM inference serving, latency/throughput tuning, profiling and benchmarking, system architecture design, mentoring
Preferred skills
CUDA kernels, ML/low-level kernels, disaggregated serving, KV caching, full SDLC experience
Technologies
SageMaker, CUDA, GPU, Inferentia, LLM frameworks
Responsibilities
Profile and tune model inference performance across hardware/software stacks; write maintainable code and participate in code reviews; collaborate with scientists and infrastructure teams to translate requirements into production solutions; evaluate emerging tools for inference optimization.
Seniority
Senior, hands-on IC