Video Evaluation, Verification & Data Engineer
Core
Build data controls, evaluation harnesses, and blinded review processes to verify generative video model improvements and ensure reproducibility and auditability.
Role type
Senior IC data engineer (generative video evaluation)
Builds
Versioned dataset manifests, frozen evaluation harnesses, and evidence packages for generative video models
Domain
Generative AI, Computer Vision, Video Production
Deliverable
production ML models
Required skills
Python, dataset versioning, experiment tracking, blinded human review design, regression testing, data lineage management, identity consistency evaluation, temporal stability analysis
Preferred skills
FiftyOne, CVAT, Label Studio, W&B, MLflow, DVC, media rights governance, evaluation gate design
Technologies
Python, FiftyOne, CVAT, Label Studio, W&B, MLflow, DVC
Responsibilities
Create versioned dataset manifests with stable IDs and checksums; Define clean development, validation, and sealed holdout sets; Build frozen evaluation harnesses for identity and temporal consistency; Combine deterministic hard gates, automated scoring, and blinded human review; Calibrate verifier thresholds against human judgments; Produce decision-ready evidence and technical handover materials
Seniority
Senior, hands-on IC
