CareerPlanSign in

Python Engineer - AI Agents

USA💼 Full-time💰 $90,000–$130,000🗓 2026-05-25 → 2026-09-26

Core

Human-in-the-loop training and evaluation of advanced generative AI agents to improve their performance, reasoning, and safety in real-world workflows.

Role type

AI Agent Training & Evaluation Specialist

Builds

Proactive, multi-step AI agents and complex orchestration workflows for client companies

Domain

Artificial Intelligence / Generative AI / Human-in-the-loop Systems

Deliverable

production ML models

Required skills

Backend engineering, AI automation, complex system integration, multi-step workflow design, API connectivity, persistent state management, security risk identification, technical feedback provision

Preferred skills

Experience with Java, JavaScript, Go, or Java, SQL databases, live environment deployment, privacy and authority escalation awareness

Technologies

Java, JavaScript, Python, SQL, Supabase, Gmail, APIs, MEMORY.md

Responsibilities

Provide human feedback to train advanced generative systems, design and coordinate complex real-world architectural workflows, evaluate system behavior in live environments, support orchestration and automation efforts, identify failures and security-sensitive behaviors in agent interactions

Seniority

Mid-level, hands-on contributor

Sourced via devitjobs · Listed on CareerPlan, which tracks 70,000+ jobs from 20+ sources.