Software Engineering, AI Research
Core
Design rigorous software engineering challenges and benchmark frontier AI models against real-world codebases to evaluate model capabilities.
Role type
AI Research Engineer (Software Engineering Benchmarks)
Builds
Benchmark datasets and evaluation pipelines for frontier AI models
Domain
Artificial Intelligence / Software Engineering
Deliverable
production ML models
Required skills
Software Engineering, Algorithms, Data Structures, Docker, Containerization, Codebase Analysis, Model Evaluation
Preferred skills
AI Research, Frontier Model Evaluation
Technologies
Docker
Responsibilities
Create complex software engineering challenges across diverse domains, Curate high-fidelity training and benchmarking data using real-world codebases, Build containerized, reproducible training pipelines, Systematically evaluate and benchmark frontier models against designed problems
