Director, Model Behavior & Evaluation Systems
Core
Own the behavioral quality system for Blue Yonder's LLM agents operating in complex supply chain workflows, defining release gates and feedback loops to ensure operational correctness and safety.
Role type
Director, Model Behavior & Evaluation Systems
Builds
LLM agents for supply chain planning, fulfillment, and delivery; evaluation infrastructure and release governance systems.
Domain
Supply Chain / Generative AI / LLM Agents
Deliverable
production ML models
Required skills
LLM product leadership, model evaluation system design, LLM tool/function calling, Python, PyTorch, Hugging Face, OpenAI Agents SDK, post-training workflows (SFT/RLHF), red-teaming, cross-functional technical leadership.
Preferred skills
Experience scaling evaluation functions at frontier labs or AI product companies.
Technologies
Python, PyTorch, Hugging Face Transformers, NVIDIA NeMo RL, OpenAI Agents SDK, Langfuse, LLM evaluation harnesses.
Responsibilities
Define launch criteria for agent behavior; establish evaluation authority as release decision infrastructure; convert traces and feedback into behavior specs and training data; lead red-teaming and behavioral risk programs; build high-performing teams for model behavior and governance.
Seniority
Director, technical leadership with hands-on depth
