微信基础-大模型评测产品经理
Core
Design and manage the evaluation system for AI products, defining metrics, standards, and processes to ensure scientific rigor and comprehensive coverage across various scenarios.
Role type
Product Manager (AI Evaluation)
Builds
Evaluation platforms, methodologies, and data-driven insights for AI/LLM products
Domain
Artificial Intelligence / Large Language Models (LLM) / NLP
Deliverable
production ML models
Required skills
AI/NLP evaluation methodology, statistical significance testing, A/B test design, PRD creation, data analysis, multi-modal evaluation, annotation system design, competitor benchmarking
Preferred skills
LLM/AI Agent product expertise, 0-1 system building experience, team management, cross-functional collaboration
Technologies
LLM, AI Agents, Multi-modal systems (image, video, voice)
Responsibilities
Design evaluation frameworks and metrics for AI products; Develop evaluation methodologies for diverse scenarios (dialogue, tool use, creative generation); Plan and design evaluation platform architecture; Analyze evaluation data to drive product and model optimization; Track competitor capabilities and industry trends; Manage annotation standards and quality control processes
Seniority
Mid-Senior, hands-on IC