🌍 Intelligence Analyst, Remote- Contract
Core
Develop and map analytic cases to taxonomies, author test cases to evaluate AI model performance on source handling and analytic rigor, and assess/grade AI-generated responses.
Role type
Contract Intelligence Analyst (AI Evaluation)
Builds
AI system benchmarks and evaluation methodologies
Domain
Artificial Intelligence / Intelligence Analysis
Deliverable
production ML models
Required skills
Analytic rigor, Source handling, Over-refusal detection, All-Source Intelligence Analysis, Structured analytic techniques, Confidence-level assessment
Preferred skills
Civilian-IC background, Teaching/mentoring experience, Nuanced uncertainty expression, Familiarity with intelligence tradecraft standards
Technologies
None stated
Responsibilities
Develop and map analytic cases to defined taxonomies, Author complex test cases designed to evaluate model performance, Assess and grade AI-generated responses, Identify and flag instances where legitimate analytic exploration is over-refused, Document edge cases and analytic disagreements, Collaborate with project leads to refine procedures for source evaluation
Seniority
Contract, hands-on IC