Senior Language Engineer
Core
Design evaluation frameworks to measure LLM technical and non-deterministic performance, implement novel prompting techniques, and spin data flywheels to extract insights for improving model response quality.
Role type
Senior Language Engineer (LLM Evaluation & Prompt Engineering)
Builds
LLM evaluation frameworks, novel prompting techniques, and data flywheels for model quality improvement
Domain
Artificial Intelligence / Large Language Models (LLMs)
Deliverable
production ML models
Required skills
LLM evaluation, prompt design, context window management, data science, machine learning, software development, agent/orchestration frameworks, tool use, product iteration, user feedback analysis
Preferred skills
Statistics, predictive analytics, consumer product shipping
Technologies
LLM evaluation frameworks, agent frameworks, orchestration frameworks
Responsibilities
Create evaluations and establish evaluation frameworks to measure technical and non-deterministic performance; implement novel prompting techniques; spin the data flywheel to extract insights from language datasets; own the status of key projects and identify risks; navigate code repositories to implement context engineering improvements and write LLM scorers and classifiers.
Seniority
Senior, hands-on IC
