LLM Engineer
Core
Providing human feedback on AI agent behavior to train advanced generative systems and optimize complex real-world architectural workflows.
Role type
Human-in-the-loop LLM evaluation engineer
Builds
Trained large language models functioning as proactive, multi-step agents
Domain
Generative AI / Autonomous Agents
Deliverable
production ML models
Required skills
backend engineering, AI automation, complex systems integration, multi-turn system interactions, multi-stage coordination workflows, agent integration with live tools, persistent state management, security vulnerability identification
Preferred skills
proficiency in multiple programming languages, SQL database experience, modular software design
Technologies
Java, JavaScript, Python, SQL, Supabase, Gmail APIs, MEMORY.md
Responsibilities
Provide human feedback on AI agent behavior, collaborate with AI teams to teach LLMs to operate as multi-step agents, design and coordinate complex architectural workflows, evaluate system outputs to improve orchestration quality, deliver technical observations to refine agent performance