微信基础-大模型评测产品经理
Core
Design and manage the evaluation system for AI products, defining metrics, standards, and processes to ensure scientific rigor and comprehensive coverage across various scenarios.
Role type
Product Manager (AI Evaluation)
Builds
Evaluation platforms, evaluation frameworks, and data-driven optimization reports for AI products.
Domain
Artificial Intelligence / Large Language Models (LLM) / NLP
Deliverable
production ML models
Required skills
AI/NLP/Dialogue system evaluation experience, statistical significance testing, A/B test design, product requirement documentation (PRD), data analysis, multi-modal evaluation methods, annotation system design, competitor benchmarking.
Preferred skills
Experience building evaluation systems from 0 to 1, LLM/AI Agent product expertise, team management for annotation or crowdsourcing.
Responsibilities
Design evaluation systems and methodologies for different AI scenarios, plan evaluation platform architecture and features, analyze evaluation data to drive product and algorithm optimization, track competitor capabilities, and manage annotation standards and quality control.
Seniority
Mid-Senior, hands-on IC
