Software Developer - Lethbridge
Core
Human-in-the-loop training and evaluation of autonomous AI agents and large language models to improve their performance in complex, real-world workflows.
Role type
AI Agent Training & Evaluation Specialist
Builds
Proactive, multi-step autonomous agents and orchestration software systems
Domain
Generative AI / Autonomous Agents
Deliverable
production ML models
Required skills
backend engineering, AI automation, complex systems integration, multi-turn system interactions, SQL databases, API integration, privacy and security analysis (prompt injection, authority escalation), persistent state management
Preferred skills
experience with multi-stage coordination tasks, creating multi-stage coordination tasks where data gathering leads to thoughtful output, integrating agents with live tools (Supabase, Gmail)
Technologies
Java, JavaScript, Python, SQL
Responsibilities
Provide human feedback to train advanced generative systems, collaborate on developing Large Language Models acting as proactive agents, design and optimize complex architectural workflows, evaluate sophisticated system behaviors, support orchestration capabilities for autonomous agents, identify edge cases and safety concerns in agent behavior
Seniority
Mid-level, hands-on contributor