AI Agent评估运营专家（智能助手方向） - AI数据与安全
Core
Design and execute evaluation frameworks for mobile assistants, personal AI Agents, and smart hardware models to ensure accuracy, comparability, and efficiency in model performance assessment.
Role type
Senior AI Agent Evaluation Specialist (Data & Safety)
Builds
Automated evaluation systems, rubrics, and end-to-end testing pipelines for complex tasks and tool usage.
Domain
AI Safety & Data Operations for LLMs and Agents
Deliverable
production ML models
Required skills
LLM/Agent fundamentals, Function Calling, Tool Use, multi-turn interaction, task planning, LLM-as-Judge, Agent-as-Judge, Python, SQL, rubric synthesis, data analysis
Preferred skills
Experience with smart hardware integration, sandbox environment construction, real-world context evaluation
Technologies
Python, SQL, LLM-as-Judge, Agent-as-Judge
Responsibilities
Design and execute evaluation schemes for mobile assistants and AI Agents; build automated evaluation systems and rubrics; explore evaluation methods for new scenarios like memory and hardware connectors; analyze model performance and drive issue resolution.
Seniority
Mid-Senior, hands-on IC
