Software Engineer - Hosted Model Infrastructure
Core
Building high-performance model serving infrastructure for AI models in production across air-gapped, edge, and enterprise environments with constrained GPU resources.
Role type
Senior IC software engineer (ML infrastructure)
Builds
Production-grade model serving platforms, deployment pipelines, and observability tools for frontier AI capabilities.
Domain
Defense, government, and enterprise sectors; Machine Learning Infrastructure
Deliverable
production ML models
Required skills
Java, C++, Python, Rust, Kubernetes, container orchestration, GPU scheduling, inference engine integration, observability, debugging complex distributed systems
Preferred skills
Python ML ecosystem familiarity, experience with air-gapped environments, security model integration
Technologies
Java, Rust, Python, Go, Docker, Kubernetes, Gradle, GitHub, GPU-accelerated inference engines
Responsibilities
Design intelligent request handling (auth, rate limiting, concurrency); build packaging/deployment pipelines for on-prem and air-gapped environments; develop observability for production AI systems; debug issues across stack (inference engines, containers, GPU drivers); run testing/benchmarking infrastructure; integrate hosted infrastructure with platform identity systems
Seniority
Senior, hands-on IC