大模型智能标注与评测算法工程师（J105767）
Core
Design, develop, and deploy an intelligent annotation and evaluation platform for large language models (LLMs), including automated data production, model calibration, and evaluation metrics.
Role type
Senior IC LLM evaluation and data engineering specialist
Builds
Automated annotation systems, Judge/Reward models, evaluation agents, and data production pipelines
Domain
Artificial Intelligence / Large Language Models / Data Engineering
Deliverable
production ML models
Required skills
Python, PyTorch, LLM evaluation methods, data engineering, model calibration, statistical significance analysis
Preferred skills
Healthcare domain experience, Agent development, RLHF, large-scale model evaluation platforms
Technologies
Python, PyTorch, LLM frameworks
Responsibilities
Design and build automated annotation and weak supervision systems; Develop Judge and Reward models for Pointwise/Pairwise/GSB evaluations; Establish evaluation rubrics and quality standards; Implement feedback-driven iteration loops for bad case analysis and model retraining; Analyze model biases (position, length, self-consistency) to improve system stability.
