Information Designer
Core
Design conversation strategies and playbooks for chatbots and voice assistants, build golden datasets for AI evaluations, and analyze AI responses at scale.
Role type
Senior Information Designer (Conversation Design & AI Evaluation)
Builds
Golden datasets, conversation principles, and evaluation frameworks for conversational AI products
Domain
AI/ML, Conversational Design, Data Analysis
Deliverable
production ML models
Required skills
Conversation design, Prompt engineering, Data analysis, User research, Stakeholder management, AI evaluation, LLM-as-a-judge modeling, Experiment design
Preferred skills
Interaction Design, HCI, Psychology, Computer Science, Knowledge of language modeling and machine learning frameworks
Technologies
Claude Code, LLM, Machine Learning, PyTorch, Python, RAG, TensorFlow, XML, HTML, Regex
Responsibilities
Identify and prioritize ways to improve customer experiences in conversational products; Create end-to-end conversation principles, design strategies, and playbooks; Build AI skills using tools to generate golden datasets for AI evaluations; Evaluate AI answers at scale through offline and online testing; Lead multiple projects, meeting deadlines and maintaining quality and business metrics
Seniority
Senior, hands-on IC with leadership