Software Development Engineer I, Inference models optimization, AWS Neuron Inference
Core
Design and implement business-critical features to optimize inference performance for a wide range of ML models and architectures on AWS Trainium custom ML accelerators.
Role type
Software Development Engineer I, Inference models optimization
Builds
High-performance kernels, distributed inference solutions, and optimized inference for LLM model families (e.g., GLM, Kimi) on AWS Trainium
Domain
Machine Learning, High-Performance Computing, Cloud Infrastructure
Deliverable
production ML models
Required skills
Data structure implementation, algorithm development, object-oriented design, C/C++, Python, Rust, systems-level development, performance profiling, debugging across hardware-software boundaries
Preferred skills
Compiler design or optimization, computer architecture, hardware simulation or emulation, ML frameworks (PyTorch, JAX), Linux kernel or embedded systems, cloud platforms (AWS), version control systems
Technologies
AWS Trainium, PyTorch, C++, Python, Rust, NKI kernels
Responsibilities
Optimize open models to run best on Trainium including sharding and developing NKI kernels; Collaborate with cross-disciplinary teams (hardware, firmware, compiler, runtime, ML) to develop software enabling custom ML accelerator silicon; Write clean, maintainable code following best practices for systems-level and performance-critical development; Participate in code reviews, contribute to technical documentation, and help deploy agentic tools; Demonstrate operational excellence through monitoring, debugging, and resolving issues across hardware-software integration boundaries; Keep learning emerging technologies in ML accelerators, compiler optimization, computer architecture, and cloud-scale distributed systems
Seniority
Junior, hands-on IC