CareerPlanSign in

Senior Software Engineer

United States, Washington, Redmond💼 Full-time🗓 2026-07-23 → 2026-09-27

Core

Designing and building major components of an AI-first, agentic-first evaluation platform to turn quality questions into rigorous, reproducible evaluations that gate what ships.

Role type

Senior Software Engineer (AI Evaluation & Agentic Systems)

Builds

An evaluation platform where autonomous agents are first-class operators, delivering services, pipelines, and tooling with observability and guardrails.

Domain

Artificial Intelligence / Autonomous Agents / Evaluation Systems

Deliverable

production ML models | infrastructure

Required skills

Large-scale distributed systems, reliability engineering, AI-first or agentic systems, AI/ML evaluation and benchmarking, .Net, Java, JavaScript, Rust, Python

Preferred skills

Modernization of evaluation runtime, cost and latency optimization, mentoring engineers, on-call reliability management

Technologies

.Net, Java, JavaScript, Rust, Python

Responsibilities

Design and build major components of the evaluation platform; implement graceful degradation, intelligent retry, and dependency-aware gating; write and review extensible, maintainable code; mentor less-experienced engineers; act as DRI for a system that gates releases; stay current with agentic systems and AI evaluation patterns.

Seniority

Senior, hands-on IC

Sourced via microsoft · Listed on CareerPlan, which tracks 813,000+ jobs from 20+ sources.