Software Development Engineer II, SageMaker AI Inference
Core
Design, build, and measure systems to optimize inference performance, reduce latency, and increase throughput for Large Language Model (LLM) serving workloads.
Role type
Senior IC software development engineer (inference optimization)
Builds
High-performance inference pipelines and kernel-level optimizations for LLMs
Domain
Artificial Intelligence / Machine Learning / Systems Engineering
Deliverable
production ML models
Required skills
C++, C#, Java, Perl, Object Oriented Design, distributed software applications, kernel-level optimization, profiling, benchmarking
Preferred skills
CUDA kernels, ML/low-level kernels, LLM architecture and inference lifecycles
Technologies
CUDA, C++, C#, Java, Perl
Responsibilities
Design and implement kernel-level optimizations and inference pipeline improvements; Profile, benchmark, and tune model inference performance; Write maintainable code and participate in code reviews; Collaborate with applied scientists and ML engineers to translate model requirements into production-ready solutions; Evaluate emerging tools and frameworks for inference optimization