Senior Software Engineer – LLM Evaluation
Core
Curate high-quality code and datasets, develop technical solutions, and evaluate AI-generated software against real-world engineering standards to improve LLM evaluation rigor.
Role type
Senior Software Engineer (LLM Evaluation)
Builds
Verification mechanisms, automated assessment agents, and reproducible coding benchmarks for AI evaluation.
Domain
Artificial Intelligence / Software Engineering
Deliverable
production ML models (via careerplan.io/jobs/a2eddfe2-a2bb-49f1-9258-dabebbba5fbb-senior-software-engineer-llm-evaluation-at-jobgether)
Required skills
Full-stack development, software architecture, system design, API design, code review, debugging, code-quality assessment, Python, JavaScript, Java, C++, Rust, Go, ReactJS
Preferred skills
Software monitoring, operational maintenance, production reliability, technical reasoning across the software lifecycle
Technologies
Python, JavaScript, ReactJS, C/C++, Java, Rust, Go
Responsibilities
Curate high-quality code examples and technical datasets for model training and benchmarking; Develop accurate solutions to software-engineering tasks; Evaluate AI-generated code for technical correctness and adherence to engineering standards; Identify implementation weaknesses and recurring coding errors; Build agents and automated mechanisms to assess code quality; Design reliable checks for consistent evaluation; Collaborate with research teams to define evaluation strategies and improve coding benchmarks.
Seniority
Senior, hands-on IC