Software Engineer - Kitchener
Core
Human-in-the-loop training and evaluation of autonomous AI agents and large language models for complex real-world workflows.
Role type
AI Training & Evaluation Specialist (Software Engineering background)
Builds
Production-like agent workflows involving data acquisition, reasoning, and output generation for leading AI organizations.
Domain
Artificial Intelligence / Generative AI / Agent Orchestration
Deliverable
production ML models
Required skills
Backend engineering, AI automation, complex system integration, multi-step system interactions, SQL databases, Python, JavaScript, Go, Java, privacy leakage detection, authority escalation detection, indirect prompt injection detection, MEMORY.md state tracking, live API integration (Supabase, Gmail), system behavior evaluation, technical feedback on sophisticated systems
Preferred skills
Multi-stage coordination workflows, connecting agents with live tools and APIs, hands-on experience with live environments
Technologies
Java, JavaScript, Python, SQL
Responsibilities
Provide high-quality human feedback on AI agent behavior to train advanced generative systems, Support projects focused on multi-step orchestration and real-world workflow optimization, Evaluate system behavior in live environments and provide detailed feedback on strengths, weaknesses, and edge cases, Review interactions for potential safety, privacy, and control issues including prompt-injection and escalation risks, Collaborate with leading AI teams to refine proactive agent performance
Seniority
Mid-level, hands-on contributor