Project Lion - Lead Prompt Engineer - United States (Remote, Part-Time)
Core
Lead a team in migrating templates to LLM autoraters, optimizing model performance, and shaping AI-driven solutions.
Role type
Senior Lead Prompt Engineer (Team Lead)
Builds
AI-driven solutions and LLM autorater systems
Domain
Artificial Intelligence / Large Language Models
Deliverable
production ML models
Required skills
Prompt Engineering, LLM tuning, team leadership, strategy, mentorship, technical documentation, data analysis (F1, precision, recall), edge case handling, tool supervision
Preferred skills
Experience with Automatic Prompt Generation (APG) tools, Automated Prompt Optimization (APO) tools, shadowbot monitoring
Technologies
APG tools, APO tools, shadowbot, LLMs
Responsibilities
Utilize APG tools to create baseline prompts for complex template clusters; Run and supervise APO tools, reviewing outputs and flagging deadlocks; Manually draft, test, and refine prompts for edge cases; Monitor shadowbot runs to track human-LLM disagreements; Run prompt versions against gold data to measure autorater quality; Draft technical launch readiness justifications
Seniority
Senior, hands-on IC with team leadership
