Software Engineer - AI Technologies - Toronto
Core
Human-in-the-loop training and evaluation of autonomous AI agents and multi-step workflows.
Role type
AI Training & Evaluation Engineer
Builds
Trained AI agents and optimized orchestration workflows for enterprise clients
Domain
Artificial Intelligence / Machine Learning Operations
Deliverable
production ML models
Required skills
backend engineering, AI automation, complex system integration, modular service design, multi-language programming (Python, JavaScript, Go, Java), SQL databases, live system debugging, multi-step workflow coordination, agent-to-tool/API integration, persistent state management, session tracking, security vulnerability detection (privacy leaks, prompt injection)
Preferred skills
designing multi-stage coordination workflows, connecting agents to live tools and APIs, spotting subtle system issues
Technologies
Java, JavaScript, Python, SQL
Responsibilities
Provide human feedback on AI agent behavior to train generative systems, evaluate autonomous multi-step agents planning and coordinating complex workflows, assess system performance in realistic architectural tasks, examine live agent interactions and identify edge cases, deliver technical observations on system behavior and reliability, identify subtle failures and weaknesses in agent workflows