Research Engineer, LangSmith Engine
Core
Build benchmarks, run experiments, and implement post-training/fine-tuning to improve the performance, cost, and efficiency of LangSmith Engine autonomous agents.
Role type
Senior Research Engineer (LLM Agents & Evaluation)
Builds
Production-ready improvements to the LangSmith Engine agent system
Domain
AI Agents, LLMs, Observability, Evaluation
Deliverable
production ML models | product features
Required skills
LLMs and AI agents, benchmark design, experimental design, post-training/fine-tuning, software engineering, system-level tradeoffs
Preferred skills
LLM-as-a-judge, automated graders, synthetic data generation, reinforcement learning, preference optimization, model serving, distributed systems
Technologies
LLMs, LangSmith, LangChain, LangGraph
Responsibilities
Build and maintain benchmarks for agent quality and efficiency; Design and run experiments to improve agent performance; Implement post-training and fine-tuning techniques; Turn successful experiments into production improvements; Define ML roadmap and mentor engineers
Seniority
Senior, hands-on IC with technical leadership
