Software Engineer II - AI/ML, Neuron Inference, Inference
Core
Design, develop, and optimize machine learning models and frameworks for deployment on custom ML hardware accelerators (AWS Trainium) to accelerate deep learning and GenAI workloads.
Role type
Senior IC machine-learning engineer (inference optimization)
Builds
Distributed inference solutions for PyTorch and high-performance kernels for ML operations on AWS Trainium
Domain
Cloud computing, machine learning, high-performance computing, hardware-software co-design
Deliverable
production ML models
Required skills
Python, system-level programming, ML model optimization, distributed computing architecture, hardware-specific optimization, performance profiling, kernel development (NKI), fusion/sharding/tiling/scheduling
Preferred skills
Experience with LLM families (GLM, GPT-OSS, Kimi), open source ecosystem collaboration, low-level optimization
Technologies
PyTorch, AWS Trainium, Neuron SDK, NKI, profiling tools
Responsibilities
Design and implement high-performance kernels for ML operations; optimize system-level performance across Neuron hardware generations; build infrastructure to onboard diverse model architectures; conduct performance analysis to resolve bottlenecks; implement optimizations like fusion and sharding; collaborate with customers to enable model performance; participate in full ML system development lifecycle from design to production deployment.