Why We Built LangSmith for Improving Agent Quality

Harrison Chase (CEO of LangChain) sits down with Bagatur (LangSmith Engineer) and Tanushree (Product Manager) for a technical roundtable on bringing production agents from prototype to rigor. They discuss the evolution of LangSmith's platform, dive deep into the new Insights Agent feature for automatically discovering patterns in production traces, and explore Multi-turn Evaluations for understanding end-to-end user interactions. 00:00 - Introductions + the evolution of LangSmith 02:39 - Introducing Insights Agent 03:49 - Real-world use cases for Insights Agent 04:44 - Customizing insights for your specific use case 05:22 - The algorithm behind Insights Agent 06:30 - The hardest part of getting Insights to work 07:13 - Tips for getting started with Insights 08:47 - Evals vs Insights - what's the difference 09:36 - What are Threads and why do they matter? 11:59 - Offline vs online evals 12:46 - Multi-turn evals for measuring agent performance in production 13:19 - Thread-level metrics and workflows 14:22 - The hot take: "Are evals dead?" 16:08 - The future of testing 17:08 - Closing thoughts Read more about our latest LangSmith updates: https://bit.ly/3WrUNDZ Learn more about Insights Agent: https://docs.langchain.com/langsmith/insights Learn more about Multi-turn Evals: https://docs.langchain.com/langsmith/online-evaluations#configure-multi-turn-online-evaluators