面向边缘的多模态开放 d1 决策模型

HuggingFace Blog · 原文 2026-10-08 · 译文 2026-10-08 · 难度 入门
精读 · 译文(非本站原创)
来自资源池 · HuggingFace Blog ↗ · 原文:Multimodal open d1 decision models for the edge ↗ · 版权归 HuggingFace Blog 所有 · 译文为个人学习的非官方翻译,以原文为准;如权利方有疑虑可联系公众号「疯了再说」删除
导读 · LiquidAI 开源多模态决策模型 d1:3B 版 16 毫秒答一问,Decision Index 10B 以下最强,600M 版带音频

为什么精读这篇Jev 点燃的「决策模型」品类出现了**开放权重 + 边缘端 + 多模态**的第一个完整答案——单次前向出决策(不产 token)、文本/图像/音频三模态、Jetson 上 16 毫秒。对 harness-model-calling:路由器的决策头从此有开源可选;对 Jev 叙事线:TypeSafe 闭源收费 vs LiquidAI 开放边缘,两条路线正式分岔。

  • d1-3B(LFM2.5-VL-3B 底座,decoder-only):Decision Index 0.2.1 上 48.57 分,10B 以下最强——压过所有 4B/9B 模型与 Decider 35B-A3B(47.11)
  • d1-omni-600M(LFM2.5-Encoder-350M 双向编码器底座 + 视觉/音频编码器):支持文本+图像或文本+音频,仅四分之一参数就超 Decider 2B(78.4 vs 77.1)——早期研究版
  • 速度:Jetson AGX Thor 16 ms / AGX Orin 26 ms / Orin Nano 50 ms 答一问;GPU 上 10 ms 内、384px 图像 18 ms 内;三问只花一问的 1.3 倍时间(AGX Thor 16→20 ms)
  • 用法:`model.system_one(state, questions)` 一次声明多个命名问题(null/choice/score 三型),一批打包 `system_one_batch` 无 padding
8 段 · 约 5 分钟

Today we're announcing open d1, our new open-weight multimodal decision models: d1-3B and d1-omni-600M. Best decision model under 10B on the Decision Index 0.2.1: d1-3B scores 48.57, ahead of every 4B and 9B model and of Decider 35B-A3B (47.11). Multimodal: d1-3B supports text and images, while d1-omni-600M supports text and images or text and audio. Fast: d1-3B answers a question in 16 ms on an NVIDIA Jetson AGX Thor, 26 ms on a Jetson AGX Orin, and 50 ms on a Jetson Orin Nano.

今天我们发布 open d1——新的开放权重多模态决策模型:d1-3B 与 d1-omni-600M。Decision Index 0.2.1 上 10B 以下最强决策模型:d1-3B 得 48.57 分,压过所有 4B 与 9B 模型以及 Decider 35B-A3B(47.11)。多模态:d1-3B 支持文本与图像,d1-omni-600M 支持文本+图像或文本+音频。快:d1-3B 在 NVIDIA Jetson AGX Thor 上 16 毫秒答一问、AGX Orin 上 26 毫秒、Orin Nano 上 50 毫秒。

How we built decision models for the edge · 我们怎么为边缘造决策模型

These open d1 decision models are built on our Liquid Foundation Models (LFMs). Unlike our generative models, decision models don't produce tokens but answer in a single forward pass. d1-3B and d1-omni-600M are trained from two very different backbones: d1-3B is trained from LFM2.5-VL-3B, our latest VLM, which is decoder-only. It accepts text and images as inputs. d1-omni-600M is trained from LFM2.5-Encoder-350M, a bidirectional encoder. It adds vision and audio encoders to handle all three modalities. It accepts either text and image, or text and audio as inputs. This model is currently in an early research release and is undergoing further development.

这些开放 d1 决策模型构建在我们的 Liquid Foundation Models(LFM)上。与生成模型不同,决策模型不产 token——单次前向直接给答案。d1-3B 与 d1-omni-600M 出自两条完全不同的底座:d1-3B 从 LFM2.5-VL-3B(我们最新的 VLM、decoder-only)训练,接受文本与图像输入;d1-omni-600M 从 LFM2.5-Encoder-350M(双向编码器)训练,加上视觉与音频编码器处理全部 3 种模态,接受文本+图像或文本+音频输入——该模型目前为早期研究版、仍在继续开发。

Benchmark results · 基准结果

We benchmarked d1-3B and d1-omni-600M on seven public datasets spanning reading comprehension, toxicity detection, intent classification, medical QA, and cross-lingual understanding. d1-3B achieves a mean score of 82.9, the highest in the table and above Decider 4B. d1-omni-600M scores 78.4, surpassing Decider 2B (77.1) with only a quarter of the parameters. We validated that d1-3B retains the vision capabilities of its LFM2.5-VL-3B backbone on standard vision benchmarks, and that d1-omni-600M handles all three modalities. We do not report any vision or audio benchmarks, as the Decision Index v0.3 includes only a private vision split and audio decision benchmarks are currently an open problem.

我们在七个公开数据集上评测 d1-3B 与 d1-omni-600M——覆盖阅读理解、毒性检测、意图分类、医疗问答与跨语言理解。d1-3B 均分 82.9 为表内最高、高于 Decider 4B;d1-omni-600M 得 78.4,以四分之一的参数超过 Decider 2B(77.1)。我们验证了 d1-3B 在标准视觉基准上保留 LFM2.5-VL-3B 底座的视觉能力、d1-omni-600M 处理全三模态——但**不报任何视觉或音频基准分数**:Decision Index v0.3 只含一个私有视觉分片,音频决策基准目前是开放问题。

Speed · 速度

In collaboration with NVIDIA, we evaluated d1-3B on the NVIDIA stack across NVIDIA GeForce RTX 4090, NVIDIA Jetson AGX Thor, Jetson AGX Orin 64 GB, and Jetson Orin Nano. Since d1-omni-600M is an early research release, we don't report any speed numbers for it in this release. Edge inference: d1-3B answers a single question in under 50 ms on every measured device. Three questions take only 1.3x the time of one, with the AGX Thor going from 16 ms to 20 ms. GPU inference: on GPU, d1-3B answers a question in under 10 ms and processes a 384px image in under 18 ms on both platforms.

与 NVIDIA 合作,我们在 NVIDIA 栈上评测 d1-3B——覆盖 GeForce RTX 4090、Jetson AGX Thor、AGX Orin 64 GB 与 Orin Nano(d1-omni-600M 属早期研究版,本release 不报它的速度数字)。边缘推理:d1-3B 在每台受测设备上 50 毫秒内答一问;三个问题只花一问的 1.3 倍时间(AGX Thor 从 16 毫秒到 20 毫秒)。GPU 推理:d1-3B 10 毫秒内答一问、两个平台上 18 毫秒内处理一张 384px 图像。

How to use open d1 decision models · 怎么用 open d1 决策模型

Reach for d1 decision models when you need fast, structured decisions, including multimodal inputs. d1-3B delivers the highest decision quality at its size, while d1-omni-600M fits where footprint matters. Install the dependencies (requires transformers>=5.14): pip install "transformers>=5.14" torch torchvision pillow. These models ship their own code, so load it with trust_remote_code=True.

需要快速、结构化的决策(含多模态输入)时伸手拿 d1 决策模型:d1-3B 在其尺寸上决策质量最高,d1-omni-600M 适合 footprint 敏感的场景。装依赖(需 transformers>=5.14):pip install "transformers>=5.14" torch torchvision pillow。模型自带代码,加载时用 trust_remote_code=True。

(示例代码要点,命令原样见原文:AutoModel.from_pretrained("LiquidAI/d1-3B", trust_remote_code=True) 加载后,model.system_one() 一次声明多个命名问题——「refund」问是否要求退款(null 型是/否)、「team」问该哪个组处理(choice 型带 billing/technical/fraud 三条 criteria)、「urgency」问多紧急(score 型带三级标尺),一段客户留言一次全答;也传 images=[photo] 直接对图像提问(数猫几只);system_one_batch() 把多条工单打包无 padding 批量决策。)

(代码原样见原文,上面是 d1-3B 的示例要点;d1-omni-600M 的跑法见其模型卡。)

Get Started · 上手

Both decision models are open-weight and available on Hugging Face today: Download d1-3B and d1-omni-600M on Hugging Face. Try: run the demos in our System One Arcade Hugging Face Space. We can't wait to see what you build.

两个决策模型都是开放权重、今天起在 HuggingFace 可下载(d1-3B 与 d1-omni-600M);试用可直接跑我们 System One Arcade 的 Hugging Face Space 演示。等不及看你做出什么。

Citation · 引用

If you use this work, please cite the release blog: @article{liquidAI2026opend1, author = {Liquid AI}, title = {Open d1: Edge decision models for text, vision, and audio}, journal = {Liquid AI Blog}, year = {2026}, note = {www.liquid.ai/blog/open-d1}}

引用请用:@article{liquidAI2026opend1, …}(Liquid AI,《Open d1: Edge decision models for text, vision, and audio》,Liquid AI Blog,2026)。

回忆挑战

先不看答案自己回忆——讲不出来再回正文找。考的是关系,不是标签。

1决策模型「不产 token、单次前向出答案」为什么在边缘设备上特别有价值?

因为生成模型的成本随输出长度线性涨(要一个个 token 自回归),而决策模型一次前向就返回结构化答案(类别/评分/置信度)——16 ms 答一问、三问只多 1.3 倍时间,这个常数成本特性正合边缘的算力与功耗预算。

2d1-3B 和 d1-omni-600M 底座完全不同,各自适合什么场景?

d1-3B 是 decoder-only VLM 底座——决策质量优先(Decision Index 10B 以下最强、保底视觉能力);omni-600M 是双向编码器加视觉/音频编码器——footprint 优先(四分之一参数超 Decider 2B,多一个音频模态但属早期研究版)。按「质量 vs 占地+模态」选。

3Jev(TypeSafe)与 d1(LiquidAI)同属「System One/决策模型」,路线差异在哪?

Jev 闭源、托管 API、按调用收费、32k 上下文;d1 开放权重、可部署到 Jetson 级边缘、多模态输入。一个卖「智能即服务」,一个卖「可拥有的决策头」——品类内部分岔从这时开始,后续看各自生态(评测集成 vs 离线部署)往哪边拉。