Lead Research Engineer – AI Evaluation & Benchmarking
Core
Lead the design and delivery of production-ready software for AI model evaluation and benchmarking, building scalable infrastructure to compare LLMs and measure quality, cost, and latency.
Role type
Senior IC lead research engineer (AI evaluation infrastructure)
Builds
Scalable evaluation infrastructure, automated evaluation pipelines, production-quality Python software, APIs, and reusable frameworks
Domain
Artificial Intelligence / Large Language Models / Cloud Engineering
Deliverable
production ML models
Required skills
Python programming, API design, automated testing, cloud engineering (AWS/Azure), SQL, technical leadership, end-to-end delivery, collaboration with research scientists, model integration and testing
Preferred skills
LLM evaluation pipeline development, experiment tracking, dataset versioning, natural language processing, cost optimization, reusable library development, open-source contributions
Technologies
Python, AWS, Azure, SQL
Responsibilities
Lead technical delivery and engineering planning; Design and build scalable evaluation infrastructure; Turn research methods into working software; Ensure reliable, reproducible model comparisons; Build practical tools for researchers and product teams; Own production reliability, performance, and responsible data use; Collaborate, mentor, and guide engineering decisions
Seniority
Senior, hands-on IC with leadership responsibilities
