Agent评估运营专家（个人助手方向） - AI数据与安全
Core
Design and execute evaluation frameworks for Personal AI Agents, focusing on capabilities like memory, proactive task execution, and tool use to ensure model performance and user experience.
Role type
Senior IC AI model evaluation specialist (Agent focus)
Builds
Automated evaluation systems, rubrics, and datasets for personal AI assistants
Domain
AI / Large Language Models / Agent Systems
Deliverable
production ML models
Required skills
LLM evaluation design, Agent-as-Judge methodology, Python, SQL, rubric synthesis, data analysis, problem attribution
Preferred skills
Experience with Function Calling, Tool Use, multi-turn interaction, task planning
Technologies
Python, SQL, LLM-as-Judge, Agent-as-Judge
Responsibilities
Design and execute evaluation schemes for Personal Agent models; Build automated evaluation systems and rubrics for core capabilities; Explore evaluation methods for proactive and personalized agents; Analyze model performance and drive issue resolution with product teams
