CareerPlan

Senior Machine Learning Engineer, Agentic Evals and Self-Improvement (Nova)

REMOTE - USFull-time2026-11-06 → 2026-11-19
Based in United States

Core

Lead evaluation and automatic optimization for Iterable Nova's AI models and agents, measuring performance and driving self-improvement loops.

Role type

Senior Machine Learning Engineer (Agentic Evals and Self-Improvement)

Builds

Evaluation systems, automated optimization loops, synthetic data pipelines, and production ML systems for Nova Intelligence.

Domain

Enterprise AI / Marketing Automation / LLM Agents

Deliverable

production ML models

Required skills

Production ML pipelines, LLM evaluation (multi-turn, tool use), automated evaluation infrastructure (LLM judges, reward models), prompt/context optimization, post-training, reinforcement learning, Python, modern ML frameworks

Preferred skills

LLM APIs (Anthropic, OpenAI, Google), automatic prompt optimization, error analysis

Technologies

Python, LLM frameworks, evaluation benchmarks, synthetic data pipelines

Responsibilities

Build evaluation systems (benchmarks, LLM judges, simulations), create diagnosis-to-fix improvement loops, optimize prompts and agent harnesses, build synthetic data pipelines, apply research to production, mentor engineers

Seniority

Senior, hands-on IC