用 LFM2.5-VL-DSpark 加速视觉语言模型

Today, we release an experimental DSpark draft model for our vision-language model (VLM) LFM2.5-VL-3B. As with our recently released LFM2.5-DSpark drafter models, it adds a speculative decoding path that trades a minimal increase in memory footprint for a larger speedup without changing output quality.

今天我们为视觉语言模型（VLM）LFM2.5-VL-3B 发布一个实验性的 DSpark 草稿模型（draft model）。与最近发布的 LFM2.5-DSpark 草稿器一样，它添加一条推测解码（speculative decoding）路径：用最小的显存占用增加，换更大的加速，且不改变输出质量。

Faster inference: decode speedups up to 3.13x on device and 2.66x on an H100, with end-to-end gains up to 2.62x and 2.27x. Small memory cost: the drafter adds 280M parameters, 8.9% on top of the 3B target. Day-one support: LFM-compatible DSpark integrations for llama.cpp, MLX-VLM, and SGLang.

更快的推理：端侧解码最高加速 3.13x、H100 上 2.66x，端到端最高 2.62x 与 2.27x。内存代价小：草稿器新增 280M 参数，在 3B 目标之上仅 8.9%。第一天支持：llama.cpp、MLX-VLM、SGLang 的 LFM 兼容 DSpark 集成。

The vision drafter uses the same architecture as our text LFM2.5-DSpark drafters: it captures the target model's hidden states at a fixed set of tapped layers and conditions on them to draft a block of k candidate tokens. Image patches and text tokens are projected into a shared representation before those layers, so the drafter operates on hidden-state vectors of identical dimensionality regardless of input modality. The inference algorithm is therefore unchanged from the text models.

视觉草稿器与文本版 LFM2.5-DSpark 同构：在固定的一组「引出层」（tapped layers）捕获目标模型的隐状态，以它们为条件起草一块 k 个候选 token。图像 patch 与文本 token 在这些层之前就被投影进共享表示，所以不管输入是什么模态，草稿器操作的都是同维度的隐状态向量——推理算法与文本版一字不改。

We follow the DSpark recipe with a mixture of vision-language SFT data, weighted toward the workloads we expect the model to serve. Based on ablations across 3, 4, and 5 layers, the draft model is a simplified attention-only drafter with 4 layers and a block size of 9. We ran 10 epochs on the final mixture and measured acceptance after each, which improved with additional training tokens before reaching diminishing returns. At inference time, we recommend a block size of 8 or 9 depending on the hardware.

训练沿用 DSpark 配方：视觉语言 SFT 混合数据、向预期 Serving 负载加权。基于 3、4、5 层的消融，草稿模型定为 4 层的简化注意力-only（attention-only）草稿器、块大小 9。最终混合数据跑了 10 个 epoch，每个后测接受率——随训练 token 增加而改善，直至收益递减。推理时建议按硬件取块大小 8 或 9。

The resulting drafter has approximately 280M parameters and increases the deployed model's parameter count by just 8.9%.

最终草稿器约 280M 参数，只把部署模型的参数量增加 8.9%。

难点 难点提示训练与架构段：数字密度最高——4 层、块 9、10 epoch、280M、8.9%，每个数都是消融或预算的结论，别当背景读。 The DSpark draft model for LFM2.5-VL-3B ships with day-one support for llama.cpp, MLX-VLM, and SGLang. We measure both on-device inference and GPU inference. Both configurations use a DSpark block size of 8 and are evaluated on six diverse vision-based tasks, including general VQA, text VQA, image captioning, chart VQA, complex reasoning, and multi-turn conversation, following the MMSpec benchmark.

LFM2.5-VL-3B 的 DSpark 草稿模型随附第一天支持：llama.cpp、MLX-VLM、SGLang。我们同时测端侧推理与 GPU 推理，两套配置都用 DSpark 块大小 8，按 MMSpec 基准在六类视觉任务上评估：通用 VQA、文本 VQA、图像描述、图表 VQA、复杂推理与多轮对话。

On-device inference. With MLX on an M5 Max, decoding runs 2.30x to 3.13x faster by task. End-to-end latency improves by 1.56x to 2.62x. With llama.cpp on an M3 Ultra, decoding improves by 1.57x to 2.14x and end-to-end by 1.30x to 1.77x.

端侧推理：MLX 跑在 M5 Max 上，解码按任务快 2.30x 到 3.13x，端到端延迟改善 1.56x 到 2.62x；llama.cpp 跑在 M3 Ultra 上，解码 1.57x 到 2.14x，端到端 1.30x 到 1.77x。

GPU inference. On H100, the same drafter delivers 20.4x to 2.66x faster decoding, with end-to-end improvements of 1.64x to 2.27x.

GPU 推理：H100 上同一个草稿器带来 20.4x 到 2.66x 的解码加速（原文如此），端到端改善 1.64x 到 2.27x。 译注 译注译注：20.4x 疑为 2.04x 之笔误——区间另一端是 2.66x，且上下文均为 2x 量级；引用此数据时建议按 2.04x 理解并向官方核实。

In LLMs, prefill is mostly compute-bound, and its cost grows (sub)quadratically with prompt length. VLMs add to this because the image first passes through a vision encoder, then the language backbone processes hundreds of visual tokens along with the text prompt. Edge devices have far less compute than datacenter GPUs, so prefill takes up more of the end-to-end latency, as time-to-first-token and decode measurements on Apple silicon and H100 show. (The M5's per-core GPU neural accelerators narrow this gap).

在 LLM 里，prefill 基本是算力受限的，成本随 prompt 长度（次）二次增长。VLM 再加一层：图像先过视觉编码器，然后语言主干要把几百个视觉 token 连同文本 prompt 一起处理。边缘设备的算力远小于数据中心 GPU，所以 prefill 占端到端延迟的比重更大——Apple 芯片与 H100 上的首 token 时间与解码测量都显示了这一点（M5 的每核 GPU 神经加速器在缩小这个差距）。

难点 难点提示局限段是全文题眼：为什么 VLM 上「解码 3x」换不来「端到端 3x」——prefill/视觉编码不在加速范围内，阿姆达尔定律收口。 Speculative decoding speeds up only decode, not vision encoding or prefill. When those stages already take up much of the wall time, even a large decode speedup gives only a modest end-to-end gain. This is Amdahl's law, where the overall speedup is capped by the part of the workload that isn't accelerated.

推测解码只加速 decode，不加速视觉编码与 prefill。当这些阶段已经占掉大半墙上时间，哪怕解码加速很大，端到端收益也有限——这就是阿姆达尔定律（Amdahl's law）：整体加速比被「没被加速的那部分负载」封顶。

Running the DSpark draft models with SGLang requires an SGLang build with DSpark support for LFM2 targets (PR #40651). Launch the target with the draft attached (python -m sglang.launch_server --model-path LiquidAI/LFM2.5-VL-3B --speculative-algorithm DSPARK --speculative-draft-model-path LiquidAI/LFM2.5-VL-3B-DSpark --speculative-draft-attention-backend flashinfer --speculative-dspark-block-size 9 --disable-radix-cache), then query the OpenAI-compatible endpoint at http://localhost:30000/v1. The block size is read from the draft's config.json; the baseline is the same command without the three --speculative-* flags.

用 SGLang 跑 DSpark 草稿模型：需要带 LFM2 目标 DSpark 支持的构建（PR #40651）。带草稿启动目标（sglang.launch_server，--model-path LiquidAI/LFM2.5-VL-3B，--speculative-algorithm DSPARK 等 --speculative-* 旗标，块大小 9），然后查 http://localhost:30000/v1 的 OpenAI 兼容端点。块大小从草稿的 config.json 读；基线就是去掉三个 --speculative-* 旗标的同一条命令。

Running them with llama.cpp requires the respective llama.cpp build (PR #29339): llama-server -m models/LFM2.5-VL-3B-F16.gguf --mmproj models/mmproj-LFM2.5-VL-3B-F16.gguf -md LFM2.5-2.6B-DSpark-F16.gguf --spec-type draft-dspark --spec-draft-n-max 8 --spec-draft-n-min 0 -fa on -ngl 99 -c 8192. Running them with MLX-VLM requires the respective build (PR #2280): mlx_vlm.server --model LiquidAI/LFM2.5-VL-3B --draft-model LiquidAI/LFM2.5-VL-3B-DSpark. The block size is read from the sidecar metadata (n-max is clamped to it). Speculative decoding is exact: the target verifies every proposed token, so greedy output equals the target alone; per-response timings report draft_n / draft_n_accepted.

llama.cpp 需要对应构建（PR #29339）：llama-server 挂 LFM2.5-VL-3B-F16 主模型与 mmproj 的 gguf、-md 挂 LFM2.5-2.6B-DSpark 草稿 gguf、--spec-type draft-dspark、--spec-draft-n-max 8、-fa on、-ngl 99、上下文 8192。MLX-VLM 需要对应构建（PR #2280）：mlx_vlm.server --model 挂 LFM2.5-VL-3B --draft-model 挂 LFM2.5-VL-3B-DSpark。块大小从边车元数据读（n-max 被钳到它）。推测解码是**精确的**：目标模型验证每个被提出的 token，贪心输出与单独跑目标完全一致；每响应计时报告 draft_n / draft_n_accepted。

Our vision DSpark draft model is available on Hugging Face in Safetensors and GGUF formats. With LFM2.5, we're delivering on our vision of AI that runs anywhere. These models are: Open-weight — download, fine-tune, and deploy without restrictions. Fast from day one — day-one support for llama.cpp, MLX, and SGLang. A complete family — from base models for customization to specialized audio and vision variants, one architecture covers diverse use cases. We can't wait to see what you build.

视觉 DSpark 草稿模型已在 HuggingFace 上提供 Safetensors 与 GGUF 两种格式。LFM2.5 承载着我们「AI 跑在任何地方」的愿景：开放权重——下载、微调、部署无限制；第一天就快——llama.cpp、MLX、SGLang 第一天支持；完整家族——从可定制的基座到专门的音频与视觉变体，一个架构覆盖多样用例。等不及看你做出什么。

For citations, please use the following reference or BibTeX: Liquid AI, "LFM2.5-VL-DSpark: Accelerating vision-language models on edge and beyond", Liquid AI Blog, Sep 2026. (@article{liquidAI2026vldspark, author = {Liquid AI}, title = {LFM2.5-VL-DSpark: Accelerating vision-language models on edge and beyond}, journal = {Liquid AI Blog}, year = {2026}, note = {www.liquid.ai/blog/lfm2-5-vl-dspark}})

引用请用以下参考或 BibTeX：Liquid AI，《LFM2.5-VL-DSpark: Accelerating vision-language models on edge and beyond》，Liquid AI Blog，2026 年 9 月。（@article{liquidAI2026vldspark, …}）