Orange Business 如何用 LangChain 与 LangSmith 建设 Live Intelligence

Orange Business is the B2B arm of Orange Group, one of Europe's largest telecommunications and digital services companies. Live Intelligence is Orange Business's flagship AI platform. It's deployed across 120,000 Orange employees and has already been adopted by 150+ customers in France & Europe.

Orange Business 是 Orange 集团（欧洲最大的电信与数字服务公司之一）的 B2B 臂膀。Live Intelligence 是它的旗舰 AI 平台：已部署覆盖 120,000 名 Orange 员工，在法国与欧洲已有 150+ 客户采用。

When Orange Business developed their first AI products, they wanted to provide customers with safe ways to develop value with AI. Live Intelligence Portal lets employees create simple AI agents and trigger basic API calls—capabilities that work for entry-level personal tasks. Orange Business wanted to build next: agents that could orchestrate complete business processes across fragmented enterprise systems, govern who can access what, and run reliably in production at scale.

Orange Business 做第一批 AI 产品时，想给客户提供安全地用 AI 创造价值的方式。Live Intelligence Portal 让员工创建简单 AI agent、触发基础 API 调用——这对入门级个人任务够用。他们想建的是下一层：能跨割裂的企业系统编排完整业务流程的 agent、能治理「谁可以访问什么」、能在生产环境规模化地可靠运行。

Reproducing the orchestration, deployment, monitoring, and observability infrastructure that enterprise-grade agents require, though, would have required over a year of development. Rather than build from scratch, Orange Business went looking for the right foundation. This is how Live Intelligence Studio was born.

然而，复现企业级 agent 所需的编排、部署、监控与可观测性基建，要一年以上的开发。与其从零造，Orange Business 出去找合适的地基——Live Intelligence Studio 就此诞生。

Orange Innovation (Orange Group's innovation scouting arm) evaluated roughly 20 major cloud providers and specialized AI vendor solutions. They selected LangChain for its model neutrality and portability—allowing the team to maintain full ownership of the agent architecture, and manage workloads in response to European data sovereignty requirements.

Orange Innovation（集团的创新侦察臂）横评了约 20 家主要云厂商与专业 AI 供应商方案。选 LangChain 是因其模型中立与可移植——团队得以完全持有 agent 架构的所有权，并按欧洲数据主权要求管理工作负载。

For evaluations and observability, LangSmith was the strongest fit. "The observability capabilities in LangSmith were significantly ahead of what we could realistically build internally," said Mathieu Ducrot, Head of AI Products at Orange Business. "Building it ourselves would likely have delayed our roadmap twelve to eighteen months."

评估与可观测性上，LangSmith 是最贴的选择。「LangSmith 的可观测能力显著领先我们现实上能内部建出的水平」，Orange Business AI 产品负责人 Mathieu Ducrot 说，「自建的话我们的路线图大概率要推迟十二到十八个月。」

Live Intelligence is structured around three components, each targeting a different user. Live Intelligence Studio is the environment for AI engineers and agent developers to design complex, cross-system workflows: an LLM-as-a-Service catalog (models hosted on Orange Business infrastructure and public cloud), an agent builder powered by LangSmith Fleet, MCP support, agent observability through LangSmith and orchestration tooling.

Live Intelligence 由三个组件构成，各面向一类用户。**Studio** 是 AI 工程师与 agent 开发者设计复杂跨系统工作流的环境：LLM-as-a-Service 模型目录（托管在 Orange Business 基础设施与公有云上）、LangSmith Fleet 驱动的 agent builder、MCP 支持、经 LangSmith 的 agent 可观测性与编排工具。

Live Intelligence Portal is the interface for business end users: HR teams drafting job descriptions, legal teams reviewing NDA clauses, software teams auto-generating test scripts, helpdesks building IT support agents. Workflows built in Live Intelligence Studio can be deployed directly into Portal, for a clean, governed end-user experience.

**Portal** 是业务端用户的界面：HR 团队起草职位描述、法务团队审 NDA 条款、软件团队自动生成测试脚本、helpdesk 搭建 IT 支持 agent。Studio 里建好的工作流可直接部署进 Portal——干净且受治理的终端体验。

Live Intelligence Console gives enterprise administrators visibility and control: usage monitoring, performance tracking, and enforcement of security and compliance policies.

**Console** 给企业管理员可见性与控制：用量监控、性能追踪、安全与合规策略的执行。

Beyond individual agents, Live Intelligence adds two organizational layers that distinguish it from simpler deployments. The Agent Registry lets teams discover, reuse, version, and govern agents and prompts across the organization. Rather than every team rebuilding the same capability, agents become shared assets with proper lifecycle management. For prompt management, Orange uses LangSmith Context Hub to maintain different prompt versions for each environment. This enables testing and validation of prompt changes before making changes available in production, and without making changes to code.

在单个 agent 之上，Live Intelligence 加了两个组织层，使它区别于简单部署。**Agent Registry** 让团队在全组织范围发现、复用、版本化并治理 agent 与提示词——不是每个团队重建同样的能力，agent 变成有正经生命周期管理的共享资产。提示词管理上，Orange 用 LangSmith Context Hub 按环境维护不同提示词版本：改动先测试验证再进生产，而且不用改代码。

The MCP Registry provides a centralized catalog of trusted tools, connectors, and external capabilities. Agents access enterprise systems (Microsoft 365, CRMs, internal APIs, and custom services) through a controlled, auditable layer.

**MCP Registry** 提供受信工具、连接器与外部能力的集中目录。agent 访问企业系统（Microsoft 365、CRM、内部 API 与自定义服务）都经过一个受控、可审计的层。

Together, these registries pull business logic into the platform layer and out of prompts so each team doesn't need to re-solve authentication, security, and governance from scratch.

两个 Registry 合起来，把业务逻辑拉回平台层、拽出提示词——每个团队不必从零重解认证、安全与治理。

Orange Business runs a 3-layer evaluation in production, built on LangSmith. RAG quality: golden datasets with question-answer pairs, expected source documents, retrieval recall, grounding quality, and answer faithfulness. Agent behavior: tool selection accuracy, task completion rates, refusal behavior, robustness against prompt injection, and adherence to business constraints. Production trace monitoring: latency, cost, model usage, failed runs, human feedback, and escalation cases.

Orange Business 在生产中跑 3 层评估（基于 LangSmith）：**RAG 质量**——带问答对、预期源文档、检索召回、接地质量与答案忠实度的黄金数据集；**agent 行为**——工具选择准确率、任务完成率、拒绝行为、对提示注入的鲁棒性、业务约束遵守；**生产链路监控**——延迟、成本、模型用量、失败运行、人工反馈与升级案例。

"Before, debugging an agent was difficult because failures were hidden inside model behavior," said Mathieu. "With LangSmith, engineers can inspect each step of the execution and understand whether the issue comes from retrieval, the prompt, the model, the tool call, or the business logic."

「以前调试 agent 很难，因为失败藏在模型行为里面」，Mathieu 说，「有了 LangSmith，工程师能检查执行的每一步，分清问题出在检索、提示词、模型、工具调用还是业务逻辑。」

With that shared visibility, AI engineers, backend engineers, product managers, and QA can look at the same traces and evaluation results rather than working from separate, disconnected views.

有了这份共享可见性，AI 工程师、后端工程师、产品经理与 QA 看的是同一些轨迹与评估结果，而不是各自隔离的视图。

Orange Business also uses LangSmith to enforce running regression evaluations before pushing changes to production. Engineers can compare agent versions, replay historical traces, and validate behavior before any deployment.

Orange Business 还用 LangSmith 强制「推送变更到生产前先跑回归评估」：工程师能对比 agent 版本、回放历史轨迹、在任何部署前验证行为。

For complex workflows with multiple agents and tool calls interacting with each other, visibility into the full execution tree enables engineers to identify and fix broken connections or issues faster. Inspections for Orange typically include: where exactly the workflow failed, which agent or tool was being executed at that point, what input was passed to that step, what response was received, how long individual steps took, how many agents and tools were executed as part of the workflow, and how the different steps contributed to the overall execution.

对多个 agent 与工具调用互相作用的复杂工作流，完整执行树的可见性让工程师更快定位断连或问题。Orange 的检查清单通常包括：工作流究竟断在哪、当时在执行哪个 agent 或工具、该步输入是什么、收到什么响应、各步耗时多久、整个工作流执行了多少 agent 与工具、各步对整体执行的贡献。

In addition, Orange builds datasets with trace examples that represent either good or bad agent behavior to use in offline evals for continuous prompt refinement.

此外，Orange 用代表好/坏 agent 行为的轨迹样本建数据集，供离线评估持续打磨提示词。

The system Orange built has improved operational efficiency across teams, giving engineers and SMEs a shared workflow for agent development and improvement.

这套系统提升了各团队的运营效率，给了工程师与业务专家（SME）一个共享的 agent 开发改进工作流。

"For us, the technical team owns the application and agentic workflow implementation, while the business team can actively participate in prompt improvement and quality evaluation. LangSmith turns prompt iteration, tracing, and evaluation into one continuous improvement workflow," said Mathieu Ducrot, Head of AI Products. Orange plans to build on that improvement loop by expanding team collaboration into online evals for even richer feedback cycles.

「对我们来说，技术团队拥有应用与 agent 工作流的实现，业务团队则能积极参与提示词改进与质量评估。LangSmith 把提示词迭代、追踪与评估变成了一个持续改进的工作流」，AI 产品负责人 Mathieu Ducrot 说。Orange 计划在这个改进环上扩展团队协作到在线评估，获得更丰富的反馈循环。

Looking ahead, the team is working on automating more complex business processes. For example, agents that support the entire Insurance claim process, with CRM and Legal workflow integrations, as well as an agent that can monitor security sensors in a port for field-engineer emergency intervention.

展望未来，团队在自动化更复杂的业务流程：比如支撑保险理赔全流程的 agent（集成 CRM 与法务工作流），以及监控港口安全传感器、触发现场工程师紧急介入的 agent。

Building an enterprise AI platform? Learn more about LangSmith and Live Intelligence.

在建企业 AI 平台？了解更多 LangSmith 与 Live Intelligence（原文有链接）。