Machine Learning Engineer / SME (LLM Eval & RAG Testing) - SME | TS/SCI
Core
Design RAG and LLM evaluation experiments, tune models on custom data, and run GPU-accelerated benchmarks to validate AI systems.
Role type
Senior IC Machine Learning Engineer (LLM Eval & RAG Testing)
Builds
AI/ML models and ensembles for mission-critical systems
Domain
Defense / AI & Machine Learning
Deliverable
production ML models
Required skills
RAG testing, LLM tuning, PyTorch/TensorFlow/Keras, GPU computing, SQL (CTEs, set ops), CI/CD (GitHub/Jenkins)
Preferred skills
Hugging Face Transformers, benchmark building, cloud architecture, Flask, text classification, XAI, ONNX
Technologies
Python, R, PyTorch, TensorFlow, Keras, SQL, GitHub, Jenkins, Tableau
Responsibilities
Design and run experiments that benchmark LLMs against defined measures; Tune neural networks/LLMs on custom datasets; Build AI/ML models and ensembles; Write complex SQL queries; Manage CI/CD pipelines for reproducible experiments.
Seniority
Senior, hands-on IC