Sr Applied Scientist, Support Agent Intelligence and Evaluation
Core
Designing evaluation science, NLP systems, and quality measurement at scale for intelligent conversational support systems serving millions of advertisers.
Role type
Senior Applied Scientist (Evaluation Science & NLP)
Builds
LLM-as-a-judge evaluation pipelines, next-generation Issue Resolution Rate (IRR) metrics, and multilingual retrieval/generation models.
Domain
AI-powered customer support for digital advertising
Deliverable
production ML models
Required skills
LLM evaluation, NLP systems, quality measurement, metric design, cross-lingual modeling, retrieval systems, generation models, latency optimization, experimental design
Preferred skills
distributed systems, online advertising domain expertise, modeling tools (R, scikit-learn, Spark, TensorFlow)
Technologies
Java, C++, Python, TensorFlow, Spark, Hadoop, numpy, scipy
Responsibilities
Design LLM-as-a-judge evaluation pipelines; develop novel metrics for advertiser resolution quality; build and improve retrieval and generation models under latency SLAs; drive multilingual science for non-English markets; partner with product and engineering to productize research
Seniority
Senior, hands-on IC