Senior Deep Learning Engineer - Model Evaluation & AI Systems
Core
Define and build evaluation methodologies and infrastructure for innovative AI models including LLMs, RAG systems, agents, and vision/multimodal models.
Role type
Senior Deep Learning Engineer (Model Evaluation & AI Systems)
Builds
NeMo Evaluator open-source platform, scalable evaluation infrastructure, harnesses, orchestration, and result pipelines
Domain
Artificial Intelligence, Large Language Models, Open Source Software
Deliverable
production ML models
Required skills
Deep learning system development, LLM behavior analysis, open-source platform building, scalable infrastructure design, technical leadership
Preferred skills
Evaluation framework design, reproducibility engineering, community engagement, cross-team collaboration
Technologies
NeMo Evaluator, GPU clusters, LLMs, RAG systems, agents, multimodal models
Responsibilities
Define evaluation methodologies for AI models, build and expand NeMo Evaluator, construct scalable evaluation infrastructure, collaborate with model training and product divisions, engage with the open-source community
Seniority
Senior, hands-on IC
