Project Lion - Senior Prompt Engineer - Japan (Remote, Part-Time)
Core
End-to-end technical migration of templates to LLM autoraters using prompt engineering to maximize model performance.
Role type
Senior Prompt Engineer (LLM evaluation & optimization)
Builds
LLM autorater systems for template clusters
Domain
AI/ML, Large Language Models, Prompt Engineering
Deliverable
production ML models
Required skills
Prompt Engineering, LLM tuning, Automated Prompt Generation (APG), Automated Prompt Optimization (APO), SQL, Data Analysis, Chain-of-thought reasoning, Few-shot learning
Preferred skills
Goose API, AI model evaluation, Computational linguistics, Linguistic expertise
Technologies
APG tools, APO tools, SQL, Shadowbot
Responsibilities
Create baseline prompts for complex template clusters; Run and supervise Automated Prompt Optimization (APO) tools; Manually draft, test, and refine prompts for edge cases; Monitor shadowbot runs to track human-LLM disagreements; Measure autorater quality against human baselines using F1, precision, and recall; Draft technical launch readiness documentation.