CareerPlan

AI Evaluation Engineer

Remote – PhilippinesRemoteFull-time2026-10-13 → 2026-11-16
Remote · from Philippines

Core

Design and maintain enterprise data environments and benchmarks to evaluate AI analysts' performance on complex, production-scale data tasks.

Role type

Senior IC AI evaluation engineer (data engineering focus)

Builds

Realistic multi-system enterprise data environments, benchmark datasets, and evaluation harnesses for AI analysts

Domain

Data engineering + AI evaluation

Deliverable

production ML models

Required skills

Advanced SQL, Data modeling, Python, Data engineering, LLM evaluation, Benchmark design, Synthetic data generation, Multi-system environment setup, Regression tracking

Preferred skills

Healthcare/Finance/Product Analytics domain knowledge, LLM evaluation frameworks, Streaming/data-lake stacks

Technologies

Snowflake, BigQuery, Redshift, Postgres, pandas, NumPy, Polars, S3, Parquet, dbt, Airflow, Spark, DuckDB

Responsibilities

Design and stand up multi-system enterprise environments; Engineer calibrated benchmark datasets; Curate task libraries and golden SQL; Own the evaluation harness and scoring integrity

Seniority

Senior, hands-on IC