AI算法工程师（自动评测方向）-即梦
Core
Building automated evaluation systems for the Jimeng Creation Agent, covering task planning, tool use, multi-turn interaction, and creative output quality across image, video, and multimodal editing scenarios.
Role type
Senior IC AI Algorithm Engineer (Agent Evaluation)
Builds
Benchmark datasets, evaluation suites, regression test systems, and automated diagnostic tools for the Jimeng Creation Agent.
Domain
Generative AI, Agentic Systems, Multimodal Content Creation
Deliverable
production ML models
Required skills
LLM/VLM evaluation, Agent Evaluation, Benchmark construction, LLM-as-Judge, Rubrics-based evaluation, trajectory analysis, tool use evaluation, bias detection, Python engineering, experimental design, data analysis, root cause analysis.
Preferred skills
Reward Model development, Function Call evaluation, AIGC image/video generation, system thinking, cross-functional collaboration.
Technologies
Python, LLM, VLM, Harness, Reward Models
Responsibilities
Design evaluation methods (LLM-as-Judge, Rubrics, Trajectory, Tool Use, Bias); Build automated diagnostic capabilities for failure attribution; Collaborate with Model, Data, and Product teams to close the iteration loop; Follow up on frontier directions like Agentic Evaluation and Multimodal Reward Models.
Seniority
Senior, hands-on IC
