CareerPlan

Machine Learning Engineer / SME (LLM Eval & RAG Testing) - SME | TS/SCI

McLean, VirginiaFull-time2026-10-28 → 2026-11-16
On-siteCitizens onlySecurity clearance required

Core

Design RAG and LLM evaluation experiments, tune models on custom data, and run GPU-accelerated benchmarks to validate AI systems.

Role type

Senior IC Machine Learning Engineer (LLM Eval & RAG Testing)

Builds

AI/ML models and ensembles for mission-critical systems

Domain

Defense / AI & Machine Learning

Deliverable

production ML models

Required skills

RAG testing, LLM tuning, PyTorch/TensorFlow/Keras, GPU computing, SQL (CTEs, set ops), CI/CD (GitHub/Jenkins)

Preferred skills

Hugging Face Transformers, benchmark building, cloud architecture, Flask, text classification, XAI, ONNX

Technologies

Python, R, PyTorch, TensorFlow, Keras, SQL, GitHub, Jenkins, Tableau

Responsibilities

Design and run experiments that benchmark LLMs against defined measures; Tune neural networks/LLMs on custom datasets; Build AI/ML models and ensembles; Write complex SQL queries; Manage CI/CD pipelines for reproducible experiments.

Seniority

Senior, hands-on IC