Senior Software Engineer – LLM Evaluation
Core
Curate high-quality code and datasets, develop software solutions, and assess AI-generated implementations against professional engineering standards to improve coding-focused AI evaluation systems.
Role type
Senior IC software engineer specializing in LLM evaluation and automated verification
Builds
Automated verification mechanisms, benchmarks, and evaluation agents for LLMs (via careerplan.io/jobs/a2036e00-9719-431c-af2f-e79755d0e82e-senior-software-engineer-llm-evaluation-at-jobgether)
Domain
Artificial Intelligence / Large Language Models / Software Engineering
Deliverable
production ML models
Required skills
Full-stack development, software architecture, system design, API design, code review, debugging, Python, JavaScript, Java, C++, Rust, ReactJS, C, Go
Preferred skills
Software monitoring, operational maintenance, production reliability, independent remote collaboration
Technologies
Python, JavaScript, Java, C++, Rust, ReactJS, C, Go
Responsibilities
Curate software examples and datasets for model training and coding benchmarks; Develop precise solutions to software-engineering tasks; Evaluate AI-generated code for technical correctness and alignment with engineering standards; Identify implementation weaknesses and recurring model-generated errors; Build agents and automated mechanisms to assess code quality; Develop reliable checks for reproducible evaluation across the software-development lifecycle; Collaborate with research teams to define evaluation strategies and improve benchmarks.
