Senior Software Engineer (Generalist, AI Systems)

## About the role We encourage you to apply to our job board directly to have an edge. Your resume is our first look into your past experiences and impact. If you have an old resume, keep it next to you for inspiration, but start with a blank document and create a resume specifically designed for this job. Align your skills and experience with the job description. Ensure you meet the minimum qualifications (MQs), and your resume clearly demonstrates alignment with the MQs. ### Role Details You'll be a senior generalist engineer at ISHI Health, shipping product features across our stack while bringing specialized depth in building evaluation and validation harnesses for AI agents and LLM-powered systems. In a lean team, you'll move fluidly between product development, platform work, and the verification infrastructure that lets us trust AI in clinical settings. You'll partner closely with the Staff Engineer, Platform on architecture decisions and with the broader team on the AI-powered features that sit at the heart of our product. ### Why this role matters ISHI's product depends on AI agents and LLM-powered workflows that surface clinical signals to clinicians and assist in care decisions. Building these features quickly is necessary; verifying that they behave correctly is non-negotiable. We need a generalist who can do both, i.e. ship product across the stack and bring the rigor that AI verification demands in a clinical context. ## What you'll do - Build and ship features across the ISHI stack (backend services, APIs, data pipelines, and integrations), partnering with the Staff Engineer, Platform and other engineers on the team. - Design and maintain evaluation harnesses for ISHI's AI agents and LLM-powered features, including prompt regression suites, retrieval quality metrics for RAG pipelines, and trajectory/behavioral testing for agentic workflows. - Contribute to AI agent and LLM feature development directly, building the agent loops, tool integrations, and retrieval systems alongside the verification infrastructure that keeps them honest. - Write strong tests as part of normal feature development (unit, integration, and end-to-end), and improve the test infrastructure where it slows the team down. - Work with AWS services across the stack, including Cognito for authentication flows and EKS for containerized service deployment. - Contribute to CI/CD pipelines, observability, and on-call practices appropriate for a clinical-grade system, in partnership with the Staff Engineer, Platform. - Help shape engineering practices that will support our FDA SaMD regulatory pathway, including verification documentation for AI components. ## What You'll Bring (MQs) - 5+ years of software engineering experience shipping production systems, with breadth across backend, infrastructure, and at least some exposure to data or ML systems. - Strong proficiency in at least one backend language (Python, Go, Java, or TypeScript/Node.js) and demonstrated comfort working across the stack.