# SGLang v0.5.16

- Product: SGLang (https://whatsnew.fyi/product/sglang)
- Vendor: LMSYS
- Date: 2026-07-25
- Version: v0.5.16
- Original notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.16
- Permalink: https://whatsnew.fyi/product/sglang/releases/v0.5.16

What's New is an index, not a publisher: every entry below links to the vendor's own release notes, which are the authoritative source. Entries are labelled where they are hand-curated sample data, pre-releases, or drawn from a secondary source such as a developer blog.

Reuse: the summaries, labels and curation here are © What's New. Quote freely with attribution and a link back; wholesale republication of the corpus is not permitted — terms: https://whatsnew.fyi/terms. The vendors' own release notes remain their publishers'.

---

- **added** — DSpark confidence-driven speculative decoding algorithm that drafts semi-autoregressively in blocks and sizes verify windows from draft confidence, enabling via --speculative-algorithm DSPARK and SGLANG_RAGGED_VERIFY_MODE=compact
- **added** — Inkling 975B-parameter multimodal MoE model support with 1M-token context, sliding-window and full attention, Mamba2 linear attention, NVFP4 MoE, and native MTP
- **added** — Support for LongCat 2.0 FP8, JetBrains Mellum v2, Pi0.5, and LongLive 2.0 diffusion model
- **added** — Linear attention on Blackwell SM100 with correct KDA MTP path and optimized recurrent_kda decode kernel
- **added** — GLM-5.2 DSA cache layer split under prefill CP that shards KV and indexer cache layers across CP ranks with --enable-dsa-cache-layer-split
- **added** — ReplaySSM Ring Spec-Verify GDN mode to reduce speculative scratch memory with --enable-gdn-replayssm-spec
- **changed** — UnifiedRadixTree is now the default for SWA, Mamba and DSA models
- **changed** — Chunked input-logprob processing is now on by default to cap peak memory
- **changed** — FA3 sparse mask kernels are off by default
- **changed** — Updated flashinfer to 0.6.14, CuTe DSL to 4.6.0, sgl-kernel to 0.4.5, and llguidance to 1.7.6
- **changed** — num_tokens_per_bs renamed to num_tokens_per_req across spec-decoding runners
- **changed** — --enable-deepep-waterfill renamed to --enable-waterfill with no deprecated alias
- **changed** — Kernels relocated to sglang.kernels namespace with updated import paths while public wrappers retain AOT sgl_kernel backend defaults
- **changed** — GDN and KDA CuteDSL prefill now fuses state I/O into the chunk-h kernel
- **removed** — Experimental QServe QoQ W4A8 and FBGEMM FP8 quantization paths
- **removed** — CUTLASS FP8 blockwise implementation for SM90 and SM100, with SM120 moved to JIT
- **removed** — --fp4-gemm-backend cutlass option and in-tree NVFP4 JIT kernels, requiring FlashInfer for NVFP4 GEMM
- **removed** — Legacy Sphinx documentation in docs/ directory with migration to Mintlify complete

#### Highlights

*574 PRs from 169 contributors.*

**DSpark: confidence-driven speculative decoding**: A new speculative algorithm. It drafts semi-autoregressively in blocks, then sizes each verify window from the draft's own confidence instead of a fixed draft length. Reaches **383.7 tok/s at accept length ~5** on DeepSeek-V4-Pro, TP8 on B300 (bs=1). Enable with `--speculative-algorithm DSPARK` and `SGLANG_RAGGED_VERIFY_MODE=compact`; tune the block with `--speculative-dspark-block-size` ([#30261](https://github.com/sgl-project/sglang/pull/30261), [#31434](https://github.com/sgl-project/sglang/pull/31434), [blog](https://www.lmsys.org/blog/2026-07-06-dspark-sglang)).

**Inkling support**: A 975B-parameter multimodal MoE with a 1M-token context. It mixes sliding-window, full and Mamba2 linear attention, and adds an NVFP4 MoE, optional vision/audio towers and native MTP. On Blackwell it reaches up to **71.7k tok/s input** and **171.0 tok/s per-user decode**. Verified on Blackwell TP4/TP8, H200 and AMD MI350X / MI355X ([#31681](https://github.com/sgl-project/sglang/pull/31681), [blog](https://www.lmsys.org/blog/2026-07-15-inkling-day0-support), [cookbook](https://docs.sglang.io/cookbook/autoregressive/ThinkingMachines/Inkling)).

**Other new models added**: [LongCat 2.0 FP8](https://docs.sglang.io/cookbook/autoregressive/Meituan/LongCat-2.0), JetBrains Mellum v2, [Pi0.5](https://docs.sglang.io/cookbook/vla/OpenPI/Pi0.5), plus diffusion support for [LongLive 2.0](https://docs.sglang.io/cookbook/diffusion/LongLive/LongLive-2.0).

**UnifiedRadixTree is now the default** for SWA, Mamba and DSA models. Replay SSM and Mamba int8 checkpoints are synced onto it, and a cache hit now resets only the state it used ([#30468](https://github.com/sgl-project/sglang/pull/30468), [#30636](https://github.com/sgl-project/sglang/pull/30636), [#30626](https://github.com/sgl-project/sglang/pull/30626), [#31643](https://github.com/sgl-project/sglang/pull/31643)).

**GLM-5.2 DSA cache layer split under prefill CP**: KV and indexer cache layers are sharded across CP ranks. Each rank owns a disjoint layer range instead of all layers. That cuts per-rank KV memory by **~74%** (0.77 to 0.20 GB/rank) at 8192 tokens on GLM-5.2-FP8, 78 layers, cp_size=4. Enable with `--enable-dsa-cache-layer-split`, which needs `--enable-prefill-cp --cp-strategy interleave` ([#29421](https://github.com/sgl-project/sglang/pull/29421)).

**ReplaySSM Ring Spec-Verify (GDN)**: Drops the per-draft SSM snapshot. Speculative scratch goes from **11.5 GB to 1.8 GB per GPU (6.4x smaller)** on Qwen3.5-35B-A3B at TP1, at accuracy and throughput parity. Opt in with `--enable-gdn-replayssm-spec` (default off; GDN with a linear draft chain only, `--speculative-eagle-topk` in {None, 1}), and tune the ring via `--linear-replayssm-cache-len` ([#28695](https://github.com/sgl-project/sglang/pull/28695)).

**Linear attention on Blackwell (SM100)**: The first correct KDA MTP path. Its `recurrent_kda` decode kernel runs at **29.6 us vs 36.8 us** for Triton (ncu, B=64). The full decode path reaches parity by B=128 and **1.35x at B=256**, and is slower below that ([#30113](https://github.com/sgl-project/sglang/pull/30113)). Separately, GDN/KDA CuteDSL prefill fuses state I/O into the chunk-h kernel ([#30169](https://github.com/sgl-project/sglang/pull/30169)).

**QServe and FBGEMM FP8 quantization are removed**: the experimental QServe (QoQ) W4A8 and FBGEMM FP8 paths are gone. `--fp4-gemm-backend cutlass` goes too, along with the in-tree NVFP4 JIT kernels, so NVFP4 GEMM now requires FlashInfer ([#31109](https://github.com/sgl-project/sglang/pull/31109), [#30448](https://github.com/sgl-project/sglang/pull/30448)).

**Dependencies**: flashinfer 0.6.14 ([#29910](https://github.com/sgl-project/sglang/pull/29910)), CuTe DSL 4.6.0 ([#31714](https://github.com/sgl-project/sglang/pull/31714)), sgl-kernel 0.4.5 ([#31496](https://github.com/sgl-project/sglang/pull/31496)), llguidance 1.7.6 ([

_[Truncated at 4000 characters — full notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.16]_