Project Lion - Senior Prompt Engineer - Italy (Remote, Part-Time)
Core
End-to-end technical migration of templates to LLM autoraters using prompt engineering to maximize model performance.
Role type
Senior Prompt Engineer (LLM migration & optimization)
Builds
LLM autorater systems for template clusters
Domain
AI/ML, Large Language Models, Prompt Engineering
Deliverable
production ML models
Required skills
Prompt Engineering, LLM tuning, Automated Prompt Generation (APG), Automated Prompt Optimization (APO), SQL, Data Analysis, Chain-of-thought reasoning, Few-shot learning
Preferred skills
Goose API, AI model evaluation, Computational linguistics, Software engineering, Linguistic expertise
Technologies
APG tools, APO tools, SQL, Goose API, LLMs
Responsibilities
Create baseline prompts for complex parent-child template clusters using APG tools; Run and supervise APO tools, reviewing outputs and flagging deadlocks; Manually draft, test, and refine prompts for edge cases; Monitor shadowbot runs to track human-LLM disagreements; Run prompt versions against gold data to measure autorater quality (F1, precision, recall); Draft technical launch readiness justifications.
Seniority
Senior, hands-on IC
