Training Example: LangChain – Review the Data, Give Your Score & Compare to the Real AI Evaluation

[H1] Powering theAgent Development Lifecycle
Make experimentation repeatable, iterate faster, and gain momentum with LangSmith.Start buildingGet a demo
BuildTestDeployMonitor
[H2] LangSmith powers top AI teams, from startups to global enterprises
[IMG: Klarna]
[IMG: Vanta]
[IMG: clay]
[IMG: RIPPLING]
[IMG: lyft]
[IMG: Gong]
[IMG: Harvey]
[IMG: ABRIDGE]
[IMG: Cloudflare]
[IMG: THE HOME DEPOT]
[IMG: workday]
[IMG: CISCO]
[IMG: Mercor]
[IMG: NU]
[IMG: monday.com]
[IMG: Nvidia]
[IMG: BRIDGEWATER]
[IMG: LinkedIn]
[IMG: coinbase]
[IMG: Klarna]
[IMG: Vanta]
[IMG: clay]
[IMG: RIPPLING]
[IMG: lyft]
[IMG: Gong]
[IMG: Harvey]
[IMG: ABRIDGE]
[IMG: Cloudflare]
[IMG: Klarna]
[IMG: Vanta]
[IMG: clay]
[IMG: RIPPLING]
[IMG: lyft]
[IMG: Gong]
[IMG: Harvey]
[IMG: ABRIDGE]
[IMG: Cloudflare]
[IMG: THE HOME DEPOT]
[IMG: workday]
[IMG: CISCO]
[IMG: Mercor]
[IMG: NU]
[IMG: monday.com]
[IMG: Nvidia]
[IMG: BRIDGEWATER]
[IMG: LinkedIn]
[IMG: coinbase]
[IMG: THE HOME DEPOT]
[IMG: workday]
[IMG: CISCO]
[IMG: Mercor]
[IMG: NU]
[IMG: monday.com]
[IMG: Nvidia]
[IMG: BRIDGEWATER]
[IMG: LinkedIn]
[IMG: coinbase]
[H3] LangSmith Agent Engineering Platform
Observe, evaluate, and deploy agents with LangSmith. LangSmith is framework-agnostic: trace your preferred framework or integrate LangSmith with any agent stack using our Python, TypeScript, Go, or Java SDKs.LangSmith EngineObservabilityEvaluationDeploymentFleetNEW RELEASE
[H4] Improve agents faster with LangSmith Engine
Surface and diagnose undetected issues autonomously to improve agents faster. LangSmith Engine clusters production failures into prioritized issues, finds the root cause in your traces and code, and proposes the fix for your review.Read the announcementObservability
[H4] Understand exactly what your agent is doing
Agents can be hard to debug and understand. Long context, branching logic, and many tools make it difficult to pinpoint where things went wrong. Tracing breaks each run into a structured timeline of steps so you can see exactly what happened, in what order, and why.Native tracing for popular agent frameworks and OpenTelemetrySDKs for Python, TypeScript, Go, and JavaMessage threading for multi-turn chat interactionsAnalytics and AI-driven insights to uncover patterns across tracesLangSmith ObservabilityEvaluation
[H4] Use real-world usage for iterative improvement
Capture production traces, turn them into test cases, and score agents with a mix of human review and automated evals. Each iteration makes your agent measurably better.Reusable LLM-as-judge and multi-turn evalsEval calibration with human feedbackHuman feedback annotationsOnline and offline scoringLangSmith EvaluationDeployment
[H4] Ship and scale agents in production
Unlike traditional web apps, agents work for long durations and need to handle async collaboration with humans and other agents. The agent server provides memory, conversational threads, and durable checkpointing out of the box - on infrastructure that’s fault-tolerant and scales to handle any workload.Supports human-in-the-loop interactions, input concurrency, and background agentsType-safe streaming of messages, UI components, and custom eventsScalable, distributed runtime to handle agent swarmsNative protocol support for A2A & MCPLangSmith DeploymentFleet
[H4] Agents for the whole company
Routine tasks like research, follow-ups, and status checks eat up your day. Describe what you need in plain language, and Fleet takes action on it across your daily tools. Turn any question or task into a recurring agent that improves with feedback and acts autonomously. Designed with enterprise security and admin in mind.Bring your own modelsUse first-party integrations or extend with any MCP serverExport agent files for pro-code developmentIntegrated LangSmith tracingAgents improve with user feedbackLangSmith Fleet
[H3] Build with our open source frameworks
Build agents fast with any model provider. Choose the right framework for the job from batteries included to low-level control.deepagents
[H5] Build intelligent agents for open-ended work
For highly autonomous, long-running agentsExplore deepagentslangchain
[H5] Quick start agents with any model provider
For building agents fast with templatesExplore langchainlanggraph
[H5] Build reliable agents with low-level control
For production agents that require some determinismExplore langgraph
[H2] Learn from teams running agents in production
More customer storiesKlarna’s AI assistant reduced case resolution time by 80% with LangSmithRead Use CaseMonday Service achieved 8.7x faster feedback loops for evals with LangSmithRead Use CasePodium reduced engineering escalations by 90% with LangSmithRead Use CaseC.H. Robinson automated 5,500 orders per day, saving 600+ hours daily with LangSmithRead Use CaseServiceNow orchestrates agents across 8 customer stages using LangSmithRead Use Case
More use cases
[H2] Trusted by the largest builder community in AI
100M+Monthly open source downloads6K+Active LangSmith customers5  Of the Fortune 10are LangSmith customers
[H2] Get started with LangSmith
Start buildingGet a demoUse LangSmith, the agent engineering platform, to improve every step of the agent development lifecycle.

[H1] Connect with our team about LangSmith
LangSmith is the agent engineering platform, built for developers and teams who need to ship reliable agents fast.‍Observe and evaluate your agentsDeploy agents without the infrastructure complexityBuild no-code agents with FleetGet in touch with our team to see how LangSmith can accelerate your agent development lifecycle. We’ll answer your questions and walk you through a tailored demo.Trusted by the best teams building agentsLooking for support? Visit our support portal here.

✓
[H2] Thanks! We'll be in touch.
Our team reviews every submission
and will reach out within 1–2 business days.

Customers
[H1] Customers choose LangChain to build reliable agents
[H2] Trusted by
Use cases in production
[IMG: Klarna]
[IMG: Vanta]
[IMG: clay]
[IMG: RIPPLING]
[IMG: lyft]
[IMG: Gong]
[IMG: Harvey]
[IMG: ABRIDGE]
[IMG: Cloudflare]
[IMG: THE HOME DEPOT]
[IMG: workday]
[IMG: CISCO]
[IMG: Mercor]
[IMG: NU]
[IMG: monday.com]
[IMG: Nvidia]
[IMG: BRIDGEWATER]
[IMG: LinkedIn]
[IMG: coinbase]
[IMG: Klarna]
[IMG: Vanta]
[IMG: clay]
[IMG: RIPPLING]
[IMG: lyft]
[IMG: Gong]
[IMG: Harvey]
[IMG: ABRIDGE]
[IMG: Cloudflare]
[IMG: Klarna]
[IMG: Vanta]
[IMG: clay]
[IMG: RIPPLING]
[IMG: lyft]
[IMG: Gong]
[IMG: Harvey]
[IMG: ABRIDGE]
[IMG: Cloudflare]
[IMG: THE HOME DEPOT]
[IMG: workday]
[IMG: CISCO]
[IMG: Mercor]
[IMG: NU]
[IMG: monday.com]
[IMG: Nvidia]
[IMG: BRIDGEWATER]
[IMG: LinkedIn]
[IMG: coinbase]
[IMG: THE HOME DEPOT]
[IMG: workday]
[IMG: CISCO]
[IMG: Mercor]
[IMG: NU]
[IMG: monday.com]
[IMG: Nvidia]
[IMG: BRIDGEWATER]
[IMG: LinkedIn]
[IMG: coinbase]
[IMG: Klarna]
[IMG: LinkedIn]
[IMG: elastic]
[IMG: coinbase]
[IMG: servicenow]
[IMG: monday.com]
[IMG: Uber]
[IMG: Klarna]
[IMG: LinkedIn]
[IMG: elastic]
[IMG: coinbase]
[IMG: servicenow]
[IMG: monday.com]
[IMG: Uber]
[IMG: Klarna]
[IMG: LinkedIn]
[IMG: elastic]
[IMG: coinbase]
[IMG: servicenow]
[IMG: monday.com]
[IMG: Uber]
[IMG: Vanta]
[IMG: exa]
[IMG: cogent]
[IMG: Serval]
[IMG: Zip]
[IMG: Listen]
[IMG: clay]
[IMG: ABRIDGE]
[IMG: Mercor]
[IMG: Harmonic]
[IMG: RIPPLING]
[IMG: Harvey]
[IMG: Vanta]
[IMG: exa]
[IMG: cogent]
[IMG: Serval]
[IMG: Zip]
[IMG: Listen]
[IMG: Vanta]
[IMG: exa]
[IMG: cogent]
[IMG: Serval]
[IMG: Zip]
[IMG: Listen]
[IMG: clay]
[IMG: ABRIDGE]
[IMG: Mercor]
[IMG: Harmonic]
[IMG: RIPPLING]
[IMG: Harvey]
[IMG: clay]
[IMG: ABRIDGE]
[IMG: Mercor]
[IMG: Harmonic]
[IMG: RIPPLING]
[IMG: Harvey]
Featured stories
[H3] Hear how engineers are shipping agents to production with LangChain's products
[H3] How Pigment built their AI business planning platform
[H3] How Rakuten speeds up time-to-market for business operations with agents
Full story
[H3] How Pagerduty built an incident management agent
Full storyMore agent engineering stories
[H3] Customer Stories
[H3] Klarna's AI Assistant speeds up customer resolution with LangSmith & LangGraph
[H3] How Podium reduced engineering intervention by 90% with LangSmith
[H3] How Rippling uses Deep Agents and LangSmith for its HR, payroll, and IT platform
[H3] ServiceNow streamlines sales and customer success operations with LangSmith
[H3] Monday.com runs evals with LangSmith for 9x faster feedback loops
[H3] How C.H. Robinson transformed logistics shipments with LangSmith & LangGraph
[H3] Pagerduty’s AI agent transforms incident data into actionable insight with LangGraph
[H3] Cisco’s platform centralizes observability using LangSmith
[H3] Unify launches agents for account qualification with LangGraph & LangSmith
[H3] Vodafone transforms data operations with AI monitoring
[H3] Trellix cuts log parsing time from days to minutes with LangSmith & LangGraph
[H3] Ready to start shipping reliable agents faster?
Deploy your agent with production-ready infrastructure. Get started in minutes with 1-click deployments, built-in APIs, and autoscaling to handle enterprise-scale traffic.Start buildingGet a demo

[H1] LangSmith Observability: AI Agent Observability Platform
[H2] Know what your agents are really doing
LangSmith Observability gives you complete visibility into agent behavior.‍Trace your preferred framework or integrate LangSmith with any agent stack using our Python, Typescript, Go, or Java SDKs.Start buildingGet a demo
[H2] Helping top teams ship great agents
Use cases in production
[IMG: Klarna]
[IMG: Vanta]
[IMG: clay]
[IMG: RIPPLING]
[IMG: lyft]
[IMG: Gong]
[IMG: Harvey]
[IMG: ABRIDGE]
[IMG: Cloudflare]
[IMG: THE HOME DEPOT]
[IMG: workday]
[IMG: CISCO]
[IMG: Mercor]
[IMG: NU]
[IMG: monday.com]
[IMG: Nvidia]
[IMG: BRIDGEWATER]
[IMG: LinkedIn]
[IMG: coinbase]
[IMG: Klarna]
[IMG: Vanta]
[IMG: clay]
[IMG: RIPPLING]
[IMG: lyft]
[IMG: Gong]
[IMG: Harvey]
[IMG: ABRIDGE]
[IMG: Cloudflare]
[IMG: Klarna]
[IMG: Vanta]
[IMG: clay]
[IMG: RIPPLING]
[IMG: lyft]
[IMG: Gong]
[IMG: Harvey]
[IMG: ABRIDGE]
[IMG: Cloudflare]
[IMG: THE HOME DEPOT]
[IMG: workday]
[IMG: CISCO]
[IMG: Mercor]
[IMG: NU]
[IMG: monday.com]
[IMG: Nvidia]
[IMG: BRIDGEWATER]
[IMG: LinkedIn]
[IMG: coinbase]
[IMG: THE HOME DEPOT]
[IMG: workday]
[IMG: CISCO]
[IMG: Mercor]
[IMG: NU]
[IMG: monday.com]
[IMG: Nvidia]
[IMG: BRIDGEWATER]
[IMG: LinkedIn]
[IMG: coinbase]
[IMG: Klarna]
[IMG: LinkedIn]
[IMG: elastic]
[IMG: coinbase]
[IMG: servicenow]
[IMG: monday.com]
[IMG: Uber]
[IMG: Klarna]
[IMG: LinkedIn]
[IMG: elastic]
[IMG: coinbase]
[IMG: servicenow]
[IMG: monday.com]
[IMG: Uber]
[IMG: Klarna]
[IMG: LinkedIn]
[IMG: elastic]
[IMG: coinbase]
[IMG: servicenow]
[IMG: monday.com]
[IMG: Uber]
[IMG: Vanta]
[IMG: exa]
[IMG: cogent]
[IMG: Serval]
[IMG: Zip]
[IMG: Listen]
[IMG: clay]
[IMG: ABRIDGE]
[IMG: Mercor]
[IMG: Harmonic]
[IMG: RIPPLING]
[IMG: Harvey]
[IMG: Vanta]
[IMG: exa]
[IMG: cogent]
[IMG: Serval]
[IMG: Zip]
[IMG: Listen]
[IMG: Vanta]
[IMG: exa]
[IMG: cogent]
[IMG: Serval]
[IMG: Zip]
[IMG: Listen]
[IMG: clay]
[IMG: ABRIDGE]
[IMG: Mercor]
[IMG: Harmonic]
[IMG: RIPPLING]
[IMG: Harvey]
[IMG: clay]
[IMG: ABRIDGE]
[IMG: Mercor]
[IMG: Harmonic]
[IMG: RIPPLING]
[IMG: Harvey]
TracingMonitoringInsightsTracing
[H3] Find failures fast with agent tracing
See exactly what your agent is doing step by step. Pinpoint the issues hurting latency, cost, and response quality.Native tracing for popular agent frameworks and OpenTelemetrySDKs for Python, TypeScript, Go, and JavaMessage threading for multi-turn chat interactionsGet started tracing your appMonitoring
[H3] Cut through the noise in production
Get a real-time view of how your agents are performing. Spot issues early, understand impact, and start triaging. LangSmith monitoring lets you score quality with online evals on the characteristics that matter the most.Cost trackingOnline LLM-as-judge and code evalsTool and agent trajectory monitoringWebhook and Pagerduty alertsLearn more about dashboardsInsights
[H3] Discover usage patterns and issues automatically
Automatically analyze and cluster your traces to detect usage patterns, common agent behaviors, and failure modes.Unsupervised topic clusteringTemplates for error analysisExecutive summary with key findingsLearn more about Insights
[H4] Search and debug traces faster with SmithDB
Agent traces are deeply nested with heavy payloads. A single conversation can generate megabytes of data across dozens of runs and tool calls. General-purpose databases can store trace data, but weren't designed for the way teams query it. SmithDB is purpose-built for agent observability.Get a demo
[H3] Designed for agent observability
[H4] Built for agent query patterns
Random access on individual runs, full-text search, JSONkey-path filtering, and trajectory queries.
[H4] Sub-second performance across millions of traces
Queries, filters, and ingestion stay fast as your trace volume grows.
[H4] Keep sensitive data in your environment
Self-host SmithDB inside your VPC so sensitive traces never leave your infrastructure. Deployment is three stateless components on object storage and Postgres. No local disks or complex sharding.Trace query12xBefore860msAfter SmithDB71msThread query9xBefore1.16sAfter SmithDB131msFull text search15xBefore6.20sAfter SmithDB400msFiltering6xBefore530msAfter SmithDB82ms
[H3] Resources for LangSmith Observability
webinar
[H5] Get started with LangSmith tracing
docs
[H5] LangSmith Observability concepts
docs
[H5] LangSmith OTel support
[H3] FAQs for LangSmith Observability
Why do teams need an LLM observability platform?Teams need an LLM observability platform to understand how their AI applications behave in production. LLM observability platforms provide visibility into RAG pipelines, AI agent decisions, track model performance metrics like cost and latency, and help debug complex failures and hallucinations by showing the complete execution trace from end-to-end.What metrics can I track in LangSmith monitoring dashboards?Custom dashboards track token usage, latency (P50, P99), error rates, cost breakdowns, and feedback scores. Configure alerts via webhooks or PagerDuty when metrics cross thresholds.What frameworks and libraries does LangSmith work with?LangSmith works with any LLM framework. Trace applications built with OpenAI SDK, Anthropic SDK, Vercel AI SDK, LlamaIndex, or custom implementations, not just LangChain. OpenTelemetry support connects to existing pipelines. Learn more.Does LangSmith support OTel?Yes. If your team has observability infrastructure on OpenTelemetry, LangSmith integrates with your existing pipelines. Send LangSmith trace data to your tools or ingest OTel data into LangSmith. See the docs.Can I use LangSmith Observability without LangSmith Evaluation?Yes. Observability and Evaluation work well together but don't require each other. Start with tracing and monitoring, then add evals when ready. For all plan types, you'll get access to both and only pay for what you use.I can’t have data leave my environment. Can I self-host LangSmith?Yes. LangSmith offers managed cloud, bring-your-own-cloud (BYOC), and self-hosted options for teams with data residency requirements. Contact us about the right option for your security needs. For more information, check out our documentation.Where is my data stored?LangSmith cloud stores data in secure infrastructure. When using LangSmith hosted at smith.langchain.com, data is stored in GCP us-central-1. If you’re on the Enterprise plan, we can deliver LangSmith to run on your kubernetes cluster in AWS, GCP, or Azure so that data never leaves your environment. For more information, check out our documentation. For teams with compliance requirements, self-hosted and BYOC options let you control where your data lives.Will LangSmith add latency to my application?No. The LangSmith SDK uses an async callback handler that sends traces to a distributed collector. Your application performance is never impacted. If LangSmith experiences an incident, your agent keeps running normally.Will you train on the data that I send LangSmith?We will not train on your data, and you own all rights to your data. See LangSmith Terms of Service for more information.How much does LangSmith cost?LangSmith has a free tier for development and small-scale production. Paid plans scale with trace volume. See our pricing page for details, or contact us for enterprise pricing.
[H3] Ready to get visibility into your agents?
LangSmith Observability is framework agnostic and works no matter how you build your agent.Start buildingGet a demo