Principal Security Research Manager
Core
Design and build end-to-end evaluation harnesses for agentic security workflows, creating benchmark suites and integrating evaluations into engineering pipelines to ensure reproducible evidence before production rollout.
Role type
Principal Security Research Manager (Agentic Security Evaluation)
Builds
Evaluation harnesses, benchmark suites, golden datasets, automated grading pipelines, and dashboards for security agent workflows.
Domain
Cybersecurity, AI Safety, Agentic Systems, Software Engineering
Deliverable
production ML models | product features | dashboards & analysis
Required skills
software engineering, system design, evaluation harness implementation, dataset creation, statistical analysis, security engineering, vulnerability management, LLM evaluation, automated grading, CI/CD integration, people management
Preferred skills
threat analysis, anomaly detection, SAST/SCA, SARIF, secure development lifecycle, experimentation design, human-review protocols
Technologies
LLMs, AI agents, automation frameworks, data pipelines, APIs, engineering telemetry, structured data, logs, traces, metrics
Responsibilities
Design and implement deterministic validators and workflow adapters for security agent evaluations; Create representative benchmark suites including adversarial and production-derived cases; Integrate evaluation results with production feedback and detect regressions; Partner with cross-functional teams to translate evaluation results into release recommendations; Convert learnings into reusable playbooks, templates, and reference implementations.
Seniority
Principal, strategy & mentorship