CareerPlan

Senior Engineer, Applied AI

New York, NYFull-time2026-06-27 → 2026-11-19
Based in United States

Core

Build production LLM-powered features for patient experience, clinical operations, and internal tooling in a vertically integrated healthcare platform.

Role type

Senior Applied AI Engineer (LLM orchestration & evaluation)

Builds

LLM application layer, orchestration, RAG, agent workflows, evaluation infrastructure, and observability systems for healthcare operations.

Domain

Healthcare technology / Large Language Models

Deliverable

production ML models

Required skills

Python, backend services, LLM APIs, prompt orchestration, RAG, embeddings, agent workflows, evaluation infrastructure, observability, safety guardrails, build-vs-buy decisions, technical leadership.

Preferred skills

LangChain, LangGraph, AgentCore, vector databases, prompt/version management, evaluation platforms (LangSmith, Braintrust, Arize Phoenix).

Responsibilities

Design and ship production LLM features end-to-end; build AI application layer including tool calling and retrieval; develop evaluation infrastructure with regression suites and human review workflows; design observability systems for accuracy and latency; build safety and reliability guardrails; integrate third-party tools and frameworks; raise engineering standards via code reviews and mentorship; collaborate with EHR operators and clinical teams to understand workflows.

Seniority

Senior, hands-on IC with technical leadership