Senior Machine Learning Engineer, Agentic Evals and Self-Improvement (Nova)
Core
Lead evaluation and automatic optimization for Iterable Nova's AI models and agents, measuring performance and driving self-improvement loops.
Role type
Senior Machine Learning Engineer (Agentic Evals and Self-Improvement)
Builds
Evaluation systems, automated optimization loops, synthetic data pipelines, and production ML systems for Nova Intelligence.
Domain
Enterprise AI / Marketing Automation / LLM Agents
Deliverable
production ML models
Required skills
Production ML pipelines, LLM evaluation (multi-turn, tool use), automated evaluation infrastructure (LLM judges, reward models), prompt/context optimization, post-training, reinforcement learning, Python, modern ML frameworks
Preferred skills
LLM APIs (Anthropic, OpenAI, Google), automatic prompt optimization, error analysis
Technologies
Python, LLM frameworks, evaluation benchmarks, synthetic data pipelines
Responsibilities
Build evaluation systems (benchmarks, LLM judges, simulations), create diagnosis-to-fix improvement loops, optimize prompts and agent harnesses, build synthetic data pipelines, apply research to production, mentor engineers
Seniority
Senior, hands-on IC