Senior/Staff Software Engineer (Back-End), Inference
Core
Design and scale the backend systems for Model Vault, Cohere's secure enterprise AI inference platform, ensuring reliability, security, and performance for millions of users.
Role type
Senior/Staff Software Engineer (Back-End, Inference)
Builds
Control-plane services for provisioning, configuring, scaling, and operating model deployments across clouds and hardware platforms.
Domain
Enterprise AI / Cloud Infrastructure / Security
Deliverable
production ML models
Required skills
Distributed systems, databases, networking, operating systems, security, Go, Python, Java, C++, Rust, concurrency, consistency, caching, queues, rate limiting, backpressure, failure domains, idempotency, graceful degradation, authentication, authorization, identity, cryptography, secrets management, tenant isolation, encryption, confidential computing, API design, incident response, capacity planning, migrations, backwards compatibility.
Preferred skills
None stated.
Technologies
Go, Python, Java, C++, Rust, Cloud Infrastructure.
Responsibilities
Evolve and scale backend systems for authentication, authorization, deployment management, billing, and metering; Lead major architectural initiatives for identity, payments, usage systems, and platform scalability; Build highly reliable control-plane services; Design APIs and distributed systems with strong guarantees; Build security and privacy into the platform architecture; Solve complex problems spanning backend systems, cloud infrastructure, security, and AI inference; Drive projects end-to-end from architecture to production operations.
Seniority
Senior/Staff, hands-on IC
