CareerPlan

Content Specialist III | AI Evaluation & Prompting

LWARemoteFull-time2026-10-03 → 2026-11-19
Remote · from United States

Core

Evaluate, refine, and improve advanced AI models and products by testing responses, writing prompts, and assessing quality against rubrics.

Role type

Senior IC AI evaluation and prompting specialist

Builds

Improved AI product experiences and communication behaviors

Domain

Artificial Intelligence / Large Language Models

Deliverable

production ML models

Required skills

AI Model Evaluation, Prompt Writing and Refinement, Writing and Editing, Rubric Based Quality Assessment, Fact Checking, Analytical Judgment

Preferred skills

Generative AI evaluation, Failure investigation, Research skills, Source verification

Technologies

Large Language Models (LLMs), Generative AI

Responsibilities

Test new AI model versions across topics and use cases, Evaluate model responses against established rubrics, Write and refine system prompts, Assess evaluation rubrics effectiveness, Perform quality reviews of human and agent evaluations, Investigate model failures and document findings

Seniority

Senior, hands-on IC