AI Reliability Engineer, HPC
Core
Ensure uptime, resiliency, and fault tolerance of HPC clusters powering MAI model training and inference.
Role type
Senior IC Site Reliability Engineer (HPC/ML Infrastructure)
Builds
Automated deployment, scaling, and failover systems for CPU+GPU environments; monitoring and alerting infrastructure.
Domain
High-Performance Computing (HPC) and Machine Learning Infrastructure
Deliverable
production ML models
Required skills
Kubernetes, Docker, CI/CD pipelines, public cloud platforms (Azure, AWS, GCP), infrastructure-as-code, monitoring and observability tools (Grafana, Datadog, OpenTelemetry), Python/Go/Bash scripting, distributed systems, networking, storage, GPU cluster operations, workload schedulers, ML training/inference pipelines, capacity planning, cost optimization.
Preferred skills
Experience with ML training or inference pipelines.
Responsibilities
Ensure uptime and resiliency of HPC clusters; design and maintain monitoring, alerting, and logging systems; build automation for deployments, incident response, scaling, and failover; lead on-call rotations and conduct postmortems; ensure data privacy and compliance; partner with ML engineers to improve developer experience.
Seniority
Senior, hands-on IC
