# SGLang changelog

> A serving runtime for large language and vision models, built around structured generation.

- Vendor: LMSYS
- Category: AI
- Official site: https://docs.sglang.ai
- Tracked by: What's New (https://whatsnew.fyi/product/sglang)
- Harvested from: GitHub (sgl-project/sglang)
- Entries below: 10 (newest first)

What's New is an index, not a publisher: every entry below links to the vendor's own release notes, which are the authoritative source. Entries are labelled where they are hand-curated sample data, pre-releases, or drawn from a secondary source such as a developer blog.

Reuse: the summaries, labels and curation here are © What's New. Quote freely with attribution and a link back; wholesale republication of the corpus is not permitted — terms: https://whatsnew.fyi/terms. The vendors' own release notes remain their publishers'.

## Releases

### v0.5.17

- Date: 2026-08-08
- Version: v0.5.17
- Original notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.17
- Permalink: https://whatsnew.fyi/product/sglang/releases/v0.5.17

- **added** — Kimi K3 day-0 support with DCP, DSpark speculative decoding, chunked-prefill PP with TP decode, KDA-aware prefix caching, HiCache L2 over DCP, LoRA on quantized weights, and reasoning, tool-call and OpenAI-compatible serving
- **added** — MiniMax-H3 video generation model support on SGLang-Diffusion with text-to-video-and-audio, first/last-frame conditioning, and image/video/audio reference conditioning
- **added** — EmbeddingGemma and LFM2.5 embedding models support
- **added** — Initial support for Rust frontend migrating network ingress and tokenized request handling to GPU scheduler from Python
- **added** — DCP communication backends with a2a, fi_a2a, and ag_rs options plus q-replicate for full-head Q projection
- **added** — DWDP for MoE prefill strategy that prefetches peer expert weights over NVLink P2P and computes all experts locally
- **added** — Session-reference-aware Unified Radix Cache for agentic and RL-rollout workloads with session_id tracking and /close_session endpoint
- **added** — SM90 FP8 MegaMoE for DeepSeek-V4 with DeepGEMM A2A path and FP8 expert weight preparation
- **added** — Cookbook recipes for Laguna-S-2.1 family and Inkling-Small models
- **changed** — Large-MoE model loading performance improved by copying oversized or non-contiguous CPU weight views into contiguous storage before H2D transfer, achieving 5.6x faster loading for DeepSeek-V4-Pro and 7.8x for GPT-OSS-20B
- **changed** — DeepSeek-V4 memory footprint on AMD reduced by removing unnecessary expert padding and moving HIP compress-state pool into memory_saver KV_CACHE region
- **changed** — Engine recovery performance improved with weight-cache daemon holding weights per GPU for faster restart recovery
- **added** — nvidia/MiniMax-M3-NVFP4 model support

#### Highlights

*582 PRs from 194 contributors.*

**Kimi K3 day-0 support**: A 2.8T-parameter multimodal LatentMoE (896 experts, top-16, routed in a 3584-dim latent space) with a 1M-token context, 69 KDA linear-attention layers interleaved with 24 MLA layers, and a MoonViT3d vision tower, shipping as a native MXFP4 checkpoint. SGLang serves it from day 0 with DCP, DSpark speculative decoding, chunked-prefill PP with TP decode, KDA-aware prefix caching, HiCache L2 over DCP, LoRA on the quantized weights, and reasoning, tool-call and OpenAI-compatible serving, verified on NVIDIA GB300 and AMD MI35x ([#32541](https://github.com/sgl-project/sglang/pull/32541), [#32828](https://github.com/sgl-project/sglang/pull/32828), [#32890](https://github.com/sgl-project/sglang/pull/32890), [#33025](https://github.com/sgl-project/sglang/pull/33025), [#33112](https://github.com/sgl-project/sglang/pull/33112), [blog](https://www.lmsys.org/blog/2026-07-27-kimi-k3-day0-support), [cookbook](https://docs.sglang.io/cookbook/autoregressive/Moonshotai/Kimi-K3), [roadmap](https://github.com/sgl-project/sglang/issues/32607)).

**MiniMax-H3 day-0 support**: MiniMax's video generation model that produces a video and a synchronized stereo audio track in one request, served natively on SGLang-Diffusion across all three public task profiles: text-to-video-and-audio (`t2va`), first/last-frame conditioning (`fl2va`), and image/video/audio reference conditioning (`ref2va`, which also covers video-to-video). Verified on B200 (TP2 + Ulysses4), H100 (TP2 + Ulysses2), AMD MI300X and MI355X (Ulysses1/2/4/8), and 2x RTX 5090 with layerwise offload ([#33275](https://github.com/sgl-project/sglang/pull/33275), [cookbook](https://docs.sglang.io/cookbook/diffusion/MiniMax/MiniMax-H3)).

**Other new models added**: [EmbeddingGemma](https://docs.sglang.io/cookbook/autoregressive/Google/EmbeddingGemma) and [LFM2.5](https://docs.sglang.io/cookbook/autoregressive/LiquidAI/LFM2.5) embedding models, nvidia/MiniMax-M3-NVFP4, plus cookbook recipes for Poolside's [Laguna-S-2.1](https://docs.sglang.io/cookbook/autoregressive/Poolside/Laguna-S-2.1) family and [Inkling-Small](https://docs.sglang.io/cookbook/autoregressive/ThinkingMachines/Inkling-Small).

**Initial support for the Rust frontend**: Migrates the front half of the server, everything from network ingress up to the point a tokenized request is handed to the GPU scheduler, from Python to a multi-threaded Rust implementation ([#29799](https://github.com/sgl-project/sglang/pull/29799)).

**DCP communication backends and q-replicate (Helix)**: The DeepSeek-MLA decode context-parallel path gains pluggable comm backends. `a2a` exchanges packed attention output plus fp32 LSE in a single NCCL collective per layer, with fp8 KV carried as uint8 byte transport; `fi_a2a` delegates the cross-rank exchange to the FlashInfer MNNVL kernel on GB200. `--dcp-replicate-q-proj` projects full-head Q locally and skips the per-layer Q head-dim all-gather. Select with `--dcp-comm-backend {ag_rs, a2a, fi_a2a}` ([#21637](https://github.com/sgl-project/sglang/pull/21637)).

**DWDP for MoE prefill**: A new prefill parallelism strategy that prefetches peer expert weights over NVLink P2P and computes all experts locally, removing EP all-to-all token dispatch. On 4x B200 with gpt-oss-120b, prefill-only, DWDP4 reaches **1.92x over DEP4** at MNT 32K / ISL 32K, and **506K vs 329K tok/s (1.54x)** at saturation (CONC=128, ISL=8K). Enable with `--dwdp-size`; the authors mark it early-development ([#29778](https://github.com/sgl-project/sglang/pull/29778)).

**Session-reference-aware Unified Radix Cache**: For agentic and RL-rollout workloads, requests can carry a stable `session_id` so eviction knows which prefixes an active session still references, instead of evicting purely by cache policy. Release the references with `/close_session`. Opt in with `--enable-session-radix-cache` ([#29173](https://github.com/sgl-project/sglang/pull/29173)).

_[Truncated at 4000 characters — full notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.17]_

### v0.5.16

- Date: 2026-07-25
- Version: v0.5.16
- Original notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.16
- Permalink: https://whatsnew.fyi/product/sglang/releases/v0.5.16

- **added** — DSpark confidence-driven speculative decoding algorithm that drafts semi-autoregressively in blocks and sizes verify windows from draft confidence, enabling via --speculative-algorithm DSPARK and SGLANG_RAGGED_VERIFY_MODE=compact
- **added** — Inkling 975B-parameter multimodal MoE model support with 1M-token context, sliding-window and full attention, Mamba2 linear attention, NVFP4 MoE, and native MTP
- **added** — Support for LongCat 2.0 FP8, JetBrains Mellum v2, Pi0.5, and LongLive 2.0 diffusion model
- **added** — Linear attention on Blackwell SM100 with correct KDA MTP path and optimized recurrent_kda decode kernel
- **added** — GLM-5.2 DSA cache layer split under prefill CP that shards KV and indexer cache layers across CP ranks with --enable-dsa-cache-layer-split
- **added** — ReplaySSM Ring Spec-Verify GDN mode to reduce speculative scratch memory with --enable-gdn-replayssm-spec
- **changed** — UnifiedRadixTree is now the default for SWA, Mamba and DSA models
- **changed** — Chunked input-logprob processing is now on by default to cap peak memory
- **changed** — FA3 sparse mask kernels are off by default
- **changed** — Updated flashinfer to 0.6.14, CuTe DSL to 4.6.0, sgl-kernel to 0.4.5, and llguidance to 1.7.6
- **changed** — num_tokens_per_bs renamed to num_tokens_per_req across spec-decoding runners
- **changed** — --enable-deepep-waterfill renamed to --enable-waterfill with no deprecated alias
- **changed** — Kernels relocated to sglang.kernels namespace with updated import paths while public wrappers retain AOT sgl_kernel backend defaults
- **changed** — GDN and KDA CuteDSL prefill now fuses state I/O into the chunk-h kernel
- **removed** — Experimental QServe QoQ W4A8 and FBGEMM FP8 quantization paths
- **removed** — CUTLASS FP8 blockwise implementation for SM90 and SM100, with SM120 moved to JIT
- **removed** — --fp4-gemm-backend cutlass option and in-tree NVFP4 JIT kernels, requiring FlashInfer for NVFP4 GEMM
- **removed** — Legacy Sphinx documentation in docs/ directory with migration to Mintlify complete

#### Highlights

*574 PRs from 169 contributors.*

**DSpark: confidence-driven speculative decoding**: A new speculative algorithm. It drafts semi-autoregressively in blocks, then sizes each verify window from the draft's own confidence instead of a fixed draft length. Reaches **383.7 tok/s at accept length ~5** on DeepSeek-V4-Pro, TP8 on B300 (bs=1). Enable with `--speculative-algorithm DSPARK` and `SGLANG_RAGGED_VERIFY_MODE=compact`; tune the block with `--speculative-dspark-block-size` ([#30261](https://github.com/sgl-project/sglang/pull/30261), [#31434](https://github.com/sgl-project/sglang/pull/31434), [blog](https://www.lmsys.org/blog/2026-07-06-dspark-sglang)).

**Inkling support**: A 975B-parameter multimodal MoE with a 1M-token context. It mixes sliding-window, full and Mamba2 linear attention, and adds an NVFP4 MoE, optional vision/audio towers and native MTP. On Blackwell it reaches up to **71.7k tok/s input** and **171.0 tok/s per-user decode**. Verified on Blackwell TP4/TP8, H200 and AMD MI350X / MI355X ([#31681](https://github.com/sgl-project/sglang/pull/31681), [blog](https://www.lmsys.org/blog/2026-07-15-inkling-day0-support), [cookbook](https://docs.sglang.io/cookbook/autoregressive/ThinkingMachines/Inkling)).

**Other new models added**: [LongCat 2.0 FP8](https://docs.sglang.io/cookbook/autoregressive/Meituan/LongCat-2.0), JetBrains Mellum v2, [Pi0.5](https://docs.sglang.io/cookbook/vla/OpenPI/Pi0.5), plus diffusion support for [LongLive 2.0](https://docs.sglang.io/cookbook/diffusion/LongLive/LongLive-2.0).

**UnifiedRadixTree is now the default** for SWA, Mamba and DSA models. Replay SSM and Mamba int8 checkpoints are synced onto it, and a cache hit now resets only the state it used ([#30468](https://github.com/sgl-project/sglang/pull/30468), [#30636](https://github.com/sgl-project/sglang/pull/30636), [#30626](https://github.com/sgl-project/sglang/pull/30626), [#31643](https://github.com/sgl-project/sglang/pull/31643)).

**GLM-5.2 DSA cache layer split under prefill CP**: KV and indexer cache layers are sharded across CP ranks. Each rank owns a disjoint layer range instead of all layers. That cuts per-rank KV memory by **~74%** (0.77 to 0.20 GB/rank) at 8192 tokens on GLM-5.2-FP8, 78 layers, cp_size=4. Enable with `--enable-dsa-cache-layer-split`, which needs `--enable-prefill-cp --cp-strategy interleave` ([#29421](https://github.com/sgl-project/sglang/pull/29421)).

**ReplaySSM Ring Spec-Verify (GDN)**: Drops the per-draft SSM snapshot. Speculative scratch goes from **11.5 GB to 1.8 GB per GPU (6.4x smaller)** on Qwen3.5-35B-A3B at TP1, at accuracy and throughput parity. Opt in with `--enable-gdn-replayssm-spec` (default off; GDN with a linear draft chain only, `--speculative-eagle-topk` in {None, 1}), and tune the ring via `--linear-replayssm-cache-len` ([#28695](https://github.com/sgl-project/sglang/pull/28695)).

**Linear attention on Blackwell (SM100)**: The first correct KDA MTP path. Its `recurrent_kda` decode kernel runs at **29.6 us vs 36.8 us** for Triton (ncu, B=64). The full decode path reaches parity by B=128 and **1.35x at B=256**, and is slower below that ([#30113](https://github.com/sgl-project/sglang/pull/30113)). Separately, GDN/KDA CuteDSL prefill fuses state I/O into the chunk-h kernel ([#30169](https://github.com/sgl-project/sglang/pull/30169)).

**QServe and FBGEMM FP8 quantization are removed**: the experimental QServe (QoQ) W4A8 and FBGEMM FP8 paths are gone. `--fp4-gemm-backend cutlass` goes too, along with the in-tree NVFP4 JIT kernels, so NVFP4 GEMM now requires FlashInfer ([#31109](https://github.com/sgl-project/sglang/pull/31109), [#30448](https://github.com/sgl-project/sglang/pull/30448)).

**Dependencies**: flashinfer 0.6.14 ([#29910](https://github.com/sgl-project/sglang/pull/29910)), CuTe DSL 4.6.0 ([#31714](https://github.com/sgl-project/sglang/pull/31714)), sgl-kernel 0.4.5 ([#31496](https://github.com/sgl-project/sglang/pull/31496)), llguidance 1.7.6 ([

_[Truncated at 4000 characters — full notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.16]_

### v0.5.15.post1

- Date: 2026-07-14
- Version: v0.5.15.post1
- Original notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.15.post1
- Permalink: https://whatsnew.fyi/product/sglang/releases/v0.5.15.post1

- **fixed** — Fix DSA model launching on non Cuda/HIP devices
- **fixed** — Fix flashinfer dependency on Cuda 12 images
- **fixed** — Fix NaN outputs caused by flashinfer trtllm FP4 MoE kernels on long input
- **fixed** — Fix GLM 5.2 IndexShare on PD disaggregation setting
- **fixed** — Fix GLM 5.2 IndexShare on Context Parallel setting

v0.5.15.post1 includes a few patches, mostly for GLM 5.2
- #30454 #30627: Fix DSA model launching on non Cuda/HIP devices
- #30858: Fix flashinfer dependency on Cuda 12 images
- #31001: Fix NaN outputs caused by flashinfer trtllm FP4 MoE kernels on long input
- #30839: Fix GLM 5.2 IndexShare on PD disaggregation setting
- #30992: Fix GLM 5.2 IndexShare on Context Parallel setting

### v0.5.15

- Date: 2026-07-10
- Version: v0.5.15
- Original notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.15
- Permalink: https://whatsnew.fyi/product/sglang/releases/v0.5.15

- **added** — Add support for Hunyuan 3 (Hy3) model
- **added** — Add support for Hierarchical Reasoning Model (HRM-Text)
- **added** — Add support for NVIDIA LocateAnything-3B model
- **added** — Add support for Baidu Unlimited-OCR model
- **added** — Add support for JoyEcho multi-shot audio/video model
- **added** — Add support for Qwen3.6 NVFP4 quantization
- **added** — Add native web search support via Exa integration
- **added** — Add FlashKDA prefill backend for safe-gate KDA linear attention
- **added** — Add ReplaySSM buffered output-only decode for linear attention
- **added** — Add FlashInfer all-to-all support for routed MoE runner
- **added** — Add decode context parallelism for MLA models including DeepSeek V3 and Kimi K2
- **changed** — Enable Spec V2 by default with zero-overhead scheduling via CUDA-graphable DSA draft-extend, dropped D2H/H2D syncs, and fused metadata ops
- **changed** — Enable Breakable CUDA Graph by default as capture path to reduce per-step kernel-launch overhead
- **changed** — Introduce IndexShare MTP to reuse indexer top-k across draft steps for up to 1.9x lower draft-step cost at long context
- **changed** — Implement TopK V2 fusing top-k selection with page-table transform supporting runtime k up to 2048
- **changed** — Fuse indexer prologue from 12 kernels to 4 for approximately 8% faster decode at batch size 1
- **changed** — Add shape-specialized JIT router GEMM and CuteDSL BF16 GEMM for Blackwell
- **changed** — Extend FlashInfer autotune to cover draft-model graphs
- **changed** — Enable FlashMLA sparse prefill by default for DeepSeek-V4 reaching over 10% throughput gain on long context
- **changed** — Add non-paged indexer support for DeepSeek-V4 long context prefill with over 5% end-to-end throughput gain
- **changed** — Upgrade transformers to 5.12.1 and upgrade tvm-ffi, sgl-deep-gemm, and tilelang dependencies

#### Highlights

**GLM-5.2 NVFP4, tuned for production**: We took time this cycle to tune GLM-5.2 NVFP4 on Blackwell for optimized production serving. It now runs at **500+ tok/s/user on 8x B300, 450 on 4x GB300** (bs=1). Run GLM-5.2 with our [cookbook](https://docs.sglang.io/cookbook/autoregressive/GLM/GLM-5.2).

  * **Spec V2 by default**: zero-overhead scheduling via CUDA-graphable DSA draft-extend, dropped D2H/H2D syncs, fused metadata ops. +11% end-to-end TPS ([#29413](https://github.com/sgl-project/sglang/pull/29413), [#29343](https://github.com/sgl-project/sglang/pull/29343), [#29166](https://github.com/sgl-project/sglang/pull/29166), [#29077](https://github.com/sgl-project/sglang/pull/29077)).
  * **IndexShare MTP**: reuses the indexer top-k across draft steps, up to 1.9x lower draft-step cost at long context ([#29959](https://github.com/sgl-project/sglang/pull/29959), [#29787](https://github.com/sgl-project/sglang/pull/29787), [#29654](https://github.com/sgl-project/sglang/pull/29654)).
  * **TopK V2**: fuses top-k selection with the page-table transform, runtime k up to 2048 ([#30274](https://github.com/sgl-project/sglang/pull/30274), [#26788](https://github.com/sgl-project/sglang/pull/26788)).
  * **Indexer prologue fusion**: 12 kernels to 4, ~8% faster decode at bs=1 ([#27705](https://github.com/sgl-project/sglang/pull/27705)).
  * **GEMM**: shape-specialized JIT router GEMM + CuteDSL BF16 GEMM for Blackwell ([#21531](https://github.com/sgl-project/sglang/pull/21531), [#30117](https://github.com/sgl-project/sglang/pull/30117)).
  * **FlashInfer autotune**: now covers draft-model graphs ([#29595](https://github.com/sgl-project/sglang/pull/29595)).


**New Model Support**: [Hunyuan 3 (Hy3)](https://docs.sglang.io/cookbook/autoregressive/Tencent/Hy3), Hierarchical Reasoning Model (HRM-Text), NVIDIA LocateAnything-3B, [Baidu Unlimited-OCR](https://docs.sglang.io/cookbook/autoregressive/Baidu/Unlimited-OCR), [JoyEcho](https://docs.sglang.io/cookbook/diffusion/JoyEcho/JoyEcho) multi-shot A/V, plus [Qwen3.6](https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.6) NVFP4 support.

**Native web search (Exa)**: Built-in `web_search` support backed by Exa ([#29342](https://github.com/sgl-project/sglang/pull/29342)).

**Breakable CUDA Graph on by default**: Breakable CUDA Graph is now the default capture path, reducing per-step kernel-launch overhead ([#29458](https://github.com/sgl-project/sglang/pull/29458)); full CUDA Graph support for the prefill phase lands as experimental ([#27988](https://github.com/sgl-project/sglang/pull/27988)).

**Linear-attention kernels (KDA / GDN)**: New FlashKDA prefill backend for safe-gate KDA linear attention ([#29472](https://github.com/sgl-project/sglang/pull/29472)), plus ReplaySSM buffered output-only decode for linear attention ([#28451](https://github.com/sgl-project/sglang/pull/28451)).

**FlashInfer A2A for routed MoE**: Adds FlashInfer all-to-all with the `flashinfer_trtllm_routed` MoE runner ([#22394](https://github.com/sgl-project/sglang/pull/22394)).

**DeepSeek-V4 Optimization**: 
* Optimizes C128 state-pool allocation using the request state pool ([#28612](https://github.com/sgl-project/sglang/pull/28612)).
* FlashMLA sparse prefill is now enabled by default for DeepSeek-V4, reaching >10% throghput gain on long context. ([#29775](https://github.com/sgl-project/sglang/pull/29775)).
* Non paged indexer support for long context prefill, with >5% e2e throughput gain.([#29619](https://github.com/sgl-project/sglang/pull/29619)).

**Decode Context Parallelism**: decode context parallelism lands for MLA models, including DeepSeek V3 and Kimi K2 series ([#14194](https://github.com/sgl-project/sglang/pull/14194)).

**Dependency upgrades**: `transformers` bumped to 5.12.1 ([#29393](https://github.com/sgl-project/sglang/pull/29393)); `tvm-ffi` / `sgl-deep-gemm` / `tilelang` upgraded ([#29554](https://github.com/sgl-project/sglang/pull/29554)).
and . See the

_[Truncated at 4000 characters — full notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.15]_

### v0.5.14

- Date: 2026-06-26
- Version: v0.5.14
- Original notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.14
- Permalink: https://whatsnew.fyi/product/sglang/releases/v0.5.14

- **added** — Support for GLM-5.2 model
- **added** — Support for LiquidAI LFM2.5 model
- **added** — Support for Kimi-K2.7-Code model
- **added** — Support for Poolside Laguna-M.1 model
- **added** — Support for DiffusionGemma model
- **added** — Support for Zyphra ZAYA1 model
- **added** — Support for MiMo-V2-ASR model
- **added** — Waterfill and LPLB MoE load balancing methods for DeepSeek expert parallelism
- **added** — KDA CuteDSL prefill kernel on Blackwell (SM100) with 1.08-1.52x speedup over Triton path
- **added** — Int8 checkpoint pool for Mamba radix cache to increase prefix-cache capacity for KDA and GDN models
- **added** — Linear-programming load balancer for MoE expert parallelism via --ep-dispatch-algorithm=lp option
- **added** — MSCCL++ integration with auto-tuned collectives for TP=8 single-node and TP=16 two-node
- **added** — FlashInfer fused allreduce, residual, and RMSNorm via --flashinfer-allreduce-fusion-backend option
- **added** — Data-parallel attention and MTP support for Nemotron-H hybrid model
- **added** — NVFP4 MoE quantization path for DeepSeek-V4 on Blackwell via --moe-runner-backend flashinfer_trtllm_routed
- **changed** — Speculative conv-window intermediate cache deduplicated with sliding-window layout, halving footprint
- **changed** — FP8 group quantization now emits power-of-two (UE8M0) scales directly from per-token group-quant kernel
- **changed** — MLA decode q-heads padded to 64 under attention-TP to dispatch cheaper head64 kernel
- **changed** — Breakable CUDA graph execution path now runs on AMD GPUs
- **fixed** — MHC prenorm kernel prewarmed at startup to remove first-run JIT slowdown on fresh server

#### Highlights

New Model Support: [GLM-5.2](https://docs.sglang.io/cookbook/autoregressive/GLM/GLM-5.2), [LiquidAI LFM2.5](https://docs.sglang.io/cookbook/autoregressive/LiquidAI/LFM2.5), [Kimi-K2.7-Code](https://docs.sglang.io/cookbook/autoregressive/Moonshotai/Kimi-K2.7-Code), [Poolside Laguna-M.1](https://docs.sglang.io/cookbook/autoregressive/Poolside/Laguna-M.1), [DiffusionGemma](https://docs.sglang.io/cookbook/autoregressive/Google/DiffusionGemma), Zyphra ZAYA1, MiMo-V2-ASR

**DeepSeek-V4 on GB300 since Day 0**: 5x higher throughput at the same interactivity, serving DeepSeek-V4 on NVIDIA GB300 with SGLang ([blog](https://pytorch.org/blog/serving-deepseek-v4-on-gb300-with-sglang-5x-higher-throughput-at-the-same-interactivity-since-day-0/)).

**Waterfill & LPLB MoE load balancing**: Two dispatch-time load-balancing methods for DeepEP expert parallelism: Waterfill for shared-expert dispatch and LPLB for redundant expert replicas, improving throughput for DeepSeek-V3/R1 and DeepSeek-V4 ([blog](https://www.lmsys.org/blog/2026-06-26-waterfill-lplb)).

**KDA CuteDSL prefill kernel on Blackwell (SM100)**: New CuteDSL prefill kernel for Kimi-Linear (KDA), 1.08-1.52x faster than the Triton path via a reusable scratch workspace, plus a cuda-graph padding fix ([#27488](https://github.com/sgl-project/sglang/pull/27488)); see the [Kimi-Linear cookbook](https://docs.sglang.io/cookbook/autoregressive/Moonshotai/Kimi-Linear).

**Linear-attention prefix-cache memory savings**: An int8 checkpoint pool stores recurrent states compactly in the Mamba radix cache, substantially increasing prefix-cache capacity for KDA / GDN models ([#28185](https://github.com/sgl-project/sglang/pull/28185)); the speculative conv-window intermediate cache is deduplicated with a sliding-window layout, halving its footprint with no numerical change ([#28302](https://github.com/sgl-project/sglang/pull/28302)).

**LPLB: linear-programming load balancer for MoE expert parallelism**: Balances token routing across redundant expert replicas by solving a per-layer LP; opt-in via `--ep-dispatch-algorithm=lp`, default behavior unchanged ([#24515](https://github.com/sgl-project/sglang/pull/24515)).

**MSCCL++ integration & MNNVL allreduce fusion**: MSCCL++ migrates to the upstream `mscclpp` Python package (Executor + DSL compiler) with auto-tuned collectives for TP=8 single-node and TP=16 two-node ([#22734](https://github.com/sgl-project/sglang/pull/22734)); FlashInfer fused allreduce + residual + RMSNorm re-enables an MNNVL backend behind `--flashinfer-allreduce-fusion-backend` (`auto` / `trtllm` / `mnnvl`), fixing the piecewise-CUDA-graph interaction ([#23402](https://github.com/sgl-project/sglang/pull/23402)).

**Nemotron DP attention + MTP**: Data-parallel attention for the hybrid Nemotron-H (Mamba2 + full attention + MoE), plus MTP support ([#24955](https://github.com/sgl-project/sglang/pull/24955)); see the [Nemotron 3 Ultra cookbook](https://docs.sglang.io/cookbook/autoregressive/NVIDIA/Nemotron3-Ultra).

**AMD: breakable CUDA graph on ROCm/HIP**: The breakable CUDA graph execution path now runs on AMD GPUs ([#28173](https://github.com/sgl-project/sglang/pull/28173)).

**NVFP4 MoE for DeepSeek-V4**: Adds an NVFP4 MoE quantization path for DeepSeek-V4 on Blackwell for higher MoE throughput; enable with `--moe-runner-backend flashinfer_trtllm_routed` ([#25820](https://github.com/sgl-project/sglang/pull/25820)); see the [DeepSeek-V4 cookbook](https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4).

**DeepSeek-V4 decode & quantization optimizations**: FP8 group quantization now emits power-of-two (UE8M0) scales directly from the per-token group-quant kernel, dropping a separate rounding pass ([#26766](https://github.com/sgl-project/sglang/pull/26766)); MLA decode q-heads are padded to 64 under attention-TP so FlashMLA dispatches the ~2x cheaper head64 kernel instead of head128 ([#27954](https://github.com/sgl-project/sglang/pull/27

_[Truncated at 4000 characters — full notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.14]_

### v0.5.13

- Date: 2026-06-13
- Version: v0.5.13
- Original notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.13
- Permalink: https://whatsnew.fyi/product/sglang/releases/v0.5.13

- **added** — Support for Nemotron 3 Ultra autoregressive model with Day-0 kernel optimizations
- **added** — Support for Step-3.7-Flash autoregressive model
- **added** — Support for Command A+ autoregressive model
- **added** — Support for Cosmos3 diffusion model
- **added** — Support for LingBot-World diffusion model
- **added** — Support for SANA-WM diffusion model
- **added** — Support for Ernie-Image diffusion model
- **added** — Support for FLUX.2-Klein 4B/9B diffusion models
- **added** — Support for Ideogram 4 diffusion model
- **changed** — Spec V2 is now the default speculative-decoding path with tree drafting production-ready across triton, FA3, MLA, and aiter backends
- **changed** — EAGLE and MTP now run on the unified V2 worker
- **changed** — Reduced per-step scheduler overhead through unified async value passing via FutureMap and prefill input transfer optimization
- **changed** — Extended piecewise and breakable CUDA graph coverage to DSA models, Kimi-K2.5, and DeepSeek V4
- **changed** — Improved Qwen 3.5 performance on Blackwell GPUs with new FlashInfer Gated DeltaNet kernels and CuTeDSL prefill kernel
- **changed** — HiCache for hybrid models (SWA/Mamba) now enabled by default through UnifiedTree
- **added** — Heterogeneous CPU plus GPU EPD disaggregation to offload VLM vision encoding onto Intel Xeon CPUs alongside GPUs
- **added** — MoRI support for disaggregated inference on AMD Instinct MI355X
- **added** — DeepSeek V4 context parallelism with MTP and sparse-attention kernels
- **added** — SGLang-Diffusion realtime video generation with msgpack frame streaming and OpenAI-style interface

##### Highlights

**New Model Support**:

- **Autoregressive**: [Nemotron 3 Ultra](https://docs.sglang.io/cookbook/autoregressive/NVIDIA/Nemotron3-Ultra) (Day-0, [blog](https://www.lmsys.org/blog/2026-06-04-nvidia-run-nemotron-3-ultra/)), [Step-3.7-Flash](https://docs.sglang.io/cookbook/autoregressive/StepFun/Step-3.7-Flash), Command A+
- **Diffusion**: [Cosmos3](https://docs.sglang.io/cookbook/diffusion/Cosmos/Cosmos3), [LingBot-World](https://docs.sglang.io/cookbook/diffusion/LingBot-World/LingBot-World), [SANA-WM](https://docs.sglang.io/cookbook/diffusion/SANA-WM/SANA-WM), [Ernie-Image](https://docs.sglang.io/cookbook/diffusion/Ernie-Image/Ernie-Image), [FLUX.2-Klein 4B/9B](https://docs.sglang.io/cookbook/diffusion/FLUX/FLUX), [Ideogram 4](https://docs.sglang.io/cookbook/diffusion/Ideogram/Ideogram4)

**Spec V2 is now the default speculative-decoding path**: Tree drafting with topk > 1 is production-ready across the triton / FA3 / MLA / aiter backends, including `page_size > 1` and Mamba/hybrid-linear models ([#26997](https://github.com/sgl-project/sglang/pull/26997), [#26972](https://github.com/sgl-project/sglang/pull/26972), [#27463](https://github.com/sgl-project/sglang/pull/27463)). Spec V1 is deprecated, with EAGLE/MTP now running on the unified V2 worker ([#25464](https://github.com/sgl-project/sglang/pull/25464)), and topk = 1 drafting is faster ([#26397](https://github.com/sgl-project/sglang/pull/26397), [#26424](https://github.com/sgl-project/sglang/pull/26424)).

**Lower per-step scheduler overhead**: Unified async value passing through FutureMap plus moving prefill input transfer onto the forward stream reduced per-step launch overhead and improved stability under high concurrency ([#25945](https://github.com/sgl-project/sglang/pull/25945), [#25879](https://github.com/sgl-project/sglang/pull/25879), [#26380](https://github.com/sgl-project/sglang/pull/26380)).

**Piecewise & Breakable CUDA Graph coverage**: Piecewise (PCG) and Breakable (BCG) CUDA Graph capture more of the model to cut per-step kernel-launch overhead, now extended to DSA models, Kimi-K2.5, and DeepSeek V4: [#23351](https://github.com/sgl-project/sglang/pull/23351), [#26382](https://github.com/sgl-project/sglang/pull/26382), [#25195](https://github.com/sgl-project/sglang/pull/25195).

**Faster Qwen 3.5 on Blackwell**: New FlashInfer Gated DeltaNet (GDN) kernels and a CuTeDSL GDN prefill kernel speed up Qwen 3.5 on Blackwell GPUs: [#22921](https://github.com/sgl-project/sglang/pull/22921), [#23273](https://github.com/sgl-project/sglang/pull/23273), [#26200](https://github.com/sgl-project/sglang/pull/26200).

**HiCache for hybrid models by default**: HybridModel (SWA/Mamba) launches HiCache through UnifiedTree by default, bringing hierarchical KV-cache offload to sliding-window and Mamba hybrids out of the box: [#27759](https://github.com/sgl-project/sglang/pull/27759).

**Heterogeneous CPU + GPU EPD disaggregation (with Intel)**: Offload VLM vision encoding onto Intel Xeon CPUs alongside GPUs, with up to ~1.3x P99 TTFT and request-throughput gains under load. ([blog](https://www.lmsys.org/blog/2026-06-01-hetero-epd/))

**MoRI on AMD Instinct MI355X (with AMD)**: Cost-competitive DeepSeek-R1 disaggregated inference via AMD's MoRI communication library, $0.169 per million tokens at 129 tok/s/user. ([blog](https://www.lmsys.org/blog/2026-05-28-mori/))

**DeepSeek V4 — context parallelism & sparse-attention kernels**: Building on the v0.5.12 Day-0 path, v0.5.13 extends DeepSeek-V4 to context-parallel serving and adds its sparse-attention kernels:

- Context Parallel + MTP: [#24934](https://github.com/sgl-project/sglang/pull/24934)
- Context Parallel + fused MoE kernel (non-DeepEP): [#24947](https://github.com/sgl-project/sglang/pull/24947)
- Sparse FlashMLA via `flash_mla_sparse_fwd`: [#25418](https://github.com/sgl-project/sglang/pull/25418)
- FP4 indexer support: [#26209](https://github.com/sgl-project/sglang/pull/26209)
- S

_[Truncated at 4000 characters — full notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.13]_

### v0.5.12.post1

- Date: 2026-05-26
- Version: v0.5.12.post1
- Original notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.12.post1
- Permalink: https://whatsnew.fyi/product/sglang/releases/v0.5.12.post1

- **fixed** — DSV4-Pro emits garbled text during single-token decode on B200/B300 by fixing deep_gemm UE8M0 scale-packing path to ceiling activation scales before packing
- **fixed** — DSV4 + EAGLE/MTP in disaggregation decode crashes around 2000 requests with SWA allocator assertion by preventing recycled KV pages from keeping stale sliding-window mappings
- **fixed** — DSV4 NSA prefill context-parallel in disaggregation prefill mode no longer crashes at scheduler startup
- **fixed** — DSV4 HiSparse with SGLANG_OPT_USE_COMPRESSOR_V2=1 restores GSM8K accuracy from 0.825 to 0.960
- **fixed** — DSV4 PD disaggregation now works with pipeline parallelism > 1 by removing stale pp_size=1 assertion
- **fixed** — DSV4-Flash with --load-format dummy and FlashInfer mxfp4 no longer hits CUDA illegal memory access during CUDA-graph capture by initializing the integer HashTopK.tid2eid lookup table in dummy load
- **fixed** — DSV4 HiCache with SGLANG_OPT_CACHE_SWA_TRANSLATION=1 no longer returns stale translation indices after cache rebuild, preventing OOB writes and wrong outputs
- **fixed** — PD NIXL disaggregation always sends aux on is_last and only expects state when truthy
- **fixed** — Fix missing group arg in get_dp_buffer
- **changed** — DSV4 warm MHC token-count buckets at startup when SGLANG_OPT_DEEPGEMM_HC_PRENORM=1 and SGLANG_OPT_USE_TILELANG_MHC_PRE=1 and hybrid SWA are enabled to eliminate 20-40s cold-bucket forward stalls
- **changed** — DSV4-Pro precompile a DeepGEMM branch for _dispatch_bf16_fp32_backend to cut runtime JIT compile cost
- **changed** — Use nvidia-cutlass-dsl with cu13 extra for CUDA 13 support required for sm_103 / B300

v0.5.12.post1 is a stability patch on top of v0.5.12. It cherry-picks 12 fixes — primarily for DeepSeek V4 — onto the release branch.

#### Bug Fixes

##### DeepSeek V4
* DSV4-Pro emits garbled text during single-token decode on B200/B300 (fix `deep_gemm` UE8M0 scale-packing path by ceiling activation scales before packing): #25733
* DSV4 + EAGLE/MTP in disaggregation decode crashes around 2000 requests with a SWA allocator assertion (recycled KV pages kept stale sliding-window mappings): #25805
* DSV4 NSA prefill context-parallel (`--enable-nsa-prefill-context-parallel --nsa-prefill-cp-mode round-robin-split`) in `--disaggregation-mode prefill`: scheduler crash at startup: #25396
* DSV4 HiSparse + `SGLANG_OPT_USE_COMPRESSOR_V2=1`: GSM8K accuracy restored from 0.825 → 0.960: #25646
* DSV4 PD disaggregation now works with pipeline parallelism > 1 (removed stale `pp_size=1` assertion): #25771
* DSV4-Flash with `--load-format dummy` + FlashInfer mxfp4 hits CUDA illegal memory access during CUDA-graph capture (the integer `HashTopK.tid2eid` lookup table was left uninitialized by dummy load): #25892
* DSV4 HiCache + `SGLANG_OPT_CACHE_SWA_TRANSLATION=1` returns stale translation indices after a cache rebuild, causing OOB writes / wrong outputs: #25889

##### Disaggregation
* [PD][NIXL] Always send aux on `is_last`; only expect state when truthy: #25699

##### Other
* Fix missing `group` arg in `get_dp_buffer`: #25585

#### Performance

* DSV4: warm MHC token-count buckets at startup (gated to `SGLANG_OPT_DEEPGEMM_HC_PRENORM=1` + `SGLANG_OPT_USE_TILELANG_MHC_PRE=1` + hybrid SWA) to eliminate 20–40s cold-bucket forward stalls: #25810
* DSV4-Pro: precompile a DeepGEMM branch for `_dispatch_bf16_fp32_backend` to cut runtime JIT compile cost: #25860

#### Dependencies

* Use `[cu13]` extra for `nvidia-cutlass-dsl` (default to CUDA 13; required for sm_103 / B300): #25576

**All PRs included in this release**: https://github.com/sgl-project/sglang/compare/v0.5.12...v0.5.12.post1

**Full Changelog**: https://github.com/sgl-project/sglang/compare/v0.5.12...v0.5.12.post1

### v0.5.12

- Date: 2026-05-16
- Version: v0.5.12
- Original notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.12
- Permalink: https://whatsnew.fyi/product/sglang/releases/v0.5.12

- **added** — DeepSeek V4 full inference path with tensor parallelism, expert parallelism, context parallelism, data parallel attention, HiSparse for CPU KV cache offloading, reasoning parser, tool call parser, DeepGemm and FlashMLA kernels
- **added** — HiCache for DeepSeek V4 under unified Radix Tree
- **added** — W4A4 MegaMoE kernels for DeepSeek V4 with faster speed and negligible accuracy drop
- **added** — Marlin and FlashInfer W4A8 MoE kernels on Hopper
- **added** — TP16 support on H100 and H20
- **added** — Pipeline Parallelism with Prefill-Decode support for DeepSeek V4
- **added** — Unified docker tag lmsysorg/sglang:v0.5.12 for all Nvidia GPUs
- **added** — TokenSpeed MLA attention backend for Blackwell with FP8 KV cache support
- **added** — Support for Intern-S2-Preview model
- **added** — Support for MiniCPM-V 4.6 model
- **added** — Support for Laguna-XS.2 model
- **added** — Support for Ring-2.6-1T model
- **added** — Support for Gemma 4 MTP model
- **added** — Adaptive Spec V2 for speculative decoding
- **added** — EAGLE-3 drafter support with Sliding Window Attention
- **added** — Kimi K2.5 EAGLE-3 MLA speculative decoding
- **added** — Gemma 3 and Gemma 4 with EAGLE-3 speculative decoding support
- **changed** — DeepSeek V3.2 and GLM-5 FP4 low-latency performance improved with PDL enablement and Cute-DSL FP4 dense GEMM
- **changed** — Gateway DeepEP source migrated from community fork to deepseek-ai/DeepEP@hybrid-ep for CUDA 13 support
- **changed** — FlashInfer pinned at version 0.6.11.post1
- **fixed** — Spec-decoding CPU overhead reduced

#### Highlights

- **DeepSeek V4 support**: Full inference path for DeepSeek-V4 (#23882), including:

    Day-0 Features: #23882
    - Parallelism: Tensor Parallelism/Expert Parallelism/Context Parallelism/Data Parallel Attention
    - Hardware: Nvidia B300/B200/H200/H100/GB200/GB300, AMD MI35X
    - Prefill-Decode Disaggregation
    - HiSparse for offloading inactive KV cache to CPU memory
    - Reasoning parser and Tool Call Parser
    - DeepGemm and FlashMLA kernels for DeepSeek V4, including MegaMoE

    Post-Day-0 additions:
    - HiCache for DeepSeek V4 under unified Radix Tree [UnifiedTree]: #24691
    - W4A4 MegaMoE kernels — faster speed with negligible accuracy drop: #25052
    - Marlin/FlashInfer W4A8 MoE kernels on Hopper: #24816 #24986
    - Faster V2 fused compression kernels: #24890
    - TP16 support on H100/H20: #24949
    - Fused SiLU+clamp+FP8 quant kernel: #24897
    - Optimized MHC + DeepGemm pipeline (fused norm, fused hc_head): #24775
    - Non-standard chat template support for DSv4: #23915
    - Multi-detokenizer support: #24944
    - Pipeline Parallelism + PD support for DeepSeek-V4: #24700
    - **A unified docker tag `lmsysorg/sglang:v0.5.12` for all Nvidia GPUs**

    See the [LMSYS blog](https://www.lmsys.org/blog/2026-04-25-deepseek-v4/) and the [DeepSeek-V4 cookbook](https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4) for more details.

- **TokenSpeed MLA attention backend (Blackwell, FP8 KV cache)**: New MLA prefill/decode kernels integrated as an attention backend on SM100, with FP8 KV cache support for low-latency MLA serving: #24925

- **DSv3.2 / GLM-5 FP4 low-latency perf**: PDL enabled across DSv3.2 / GLM-5 kernels, `torch.mm` for the DeepSeek V3.2 indexer GEMM, and a reland of the Cute-DSL FP4 dense GEMM — materially trimming low-latency overheads on FP4 paths: #23965, #23856, #23590, #25311

- **New Model Support**: DeepSeek V4 #23882, Intern-S2-Preview #24875, MiniCPM-V 4.6 #24855, Laguna-XS.2 #24204, Ring-2.6-1T #25360, and Gemma 4 MTP #24436 — with cookbook recipes for tuned deployment commands. See [docs.sglang.io/cookbook](https://docs.sglang.io/cookbook)

- **HiCache + UnifiedRadixTree**: HiCache framework support for UnifiedRadixTree (with SWA), HiCache for DeepSeek V4, SSD offload through Mooncake store, and stability fixes across cascade eviction, tombstone replay, and partial-match paths: #23316, #23391, #24691, #24277, #24943, #24972, #25068, #25277

- **Speculative Decoding V2 maturation**: Adaptive Spec V2, EAGLE-3 SWA + newer drafters, Kimi K2.5 EAGLE-3 MLA, Gemma 3/4 + EAGLE-3, and an extensive naming / shape-handling refactor across draft-extend paths: #23336, #24663, #24664, #24826, #23976, #24859

- **CUDA 13 DeepEP migration**: Gateway DeepEP source swapped from a community fork to `deepseek-ai/DeepEP@hybrid-ep` so DeepEP builds and runs cleanly on the CUDA 13 default; FlashInfer pinned at 0.6.11.post1 alongside a gpt-oss triton-kernel fix: #25113


##### New Model Support

Entries with a published cookbook recipe come first; entries whose cookbook page is still pending are grouped at the bottom.

* DeepSeek V4 (see [cookbook](https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4); [LMSYS blog](https://www.lmsys.org/blog/2026-04-25-deepseek-v4/))
* Intern-S2-Preview: #24875, #25115, #25134 (see [cookbook](https://docs.sglang.io/cookbook/autoregressive/InternLM/Intern-S2-Preview))
* MiniCPM-V 4.6: #24855, #24876, #24991, #24998 (see [cookbook](https://docs.sglang.io/cookbook/autoregressive/OpenBMB/MiniCPM-V-4_6))
* Laguna-XS.2 (Poolside): #24204, #24730 (see [cookbook](https://docs.sglang.io/cookbook/autoregressive/Poolside/Laguna-XS.2))
* Ring-2.6-1T (InclusionAI, trillion-param reasoning): #25360, #25370 (see [cookbook](https://docs.sglang.io/cookbook/autoregressive/InclusionAI/Ring-2.6-1T))
* Gemma 4 MTP (MTP head for Gemma 4): #24436, #24433
* Trinity-mini (Ascend NPU, ~90% accuracy): #

_[Truncated at 4000 characters — full notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.12]_

### v0.5.11

- Date: 2026-05-05
- Version: v0.5.11
- Original notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.11
- Permalink: https://whatsnew.fyi/product/sglang/releases/v0.5.11

- **changed** — Default CUDA version upgraded to 13.0 across SGLang, sgl-kernel, and Docker images, with PyTorch upgraded from 2.9 to 2.11
- **changed** — Speculative Decoding V2 with overlap scheduling is now the default, reducing per-step CPU cost for EAGLE/MTP/DFLASH paths
- **added** — Decode-side prefix caching now works under prefill/decode disaggregation, recovering radix-cache hit rates and TTFT savings for long shared prefixes in disaggregated deployments
- **added** — Support for Gemma 4, GLM-5.1, Qwen3.6, MiMo-V2.5/V2.5-Pro, Ling-2.6-Flash, Mistral Medium 3.5, and Kimi-K2.6 models
- **added** — DFLASH speculative decoding kernel with support across model backends and AMD ROCm
- **added** — FA3 kernels integrated alongside FA4 as a high-performance option
- **added** — LoRA support for DeepSeek-V3 and Kimi-K2, enabling adapter-based fine-tuning of MLA-based MoE models
- **added** — All-reduce and RMSNorm fusion under context parallel for end-to-end speedups
- **added** — Support for moe_dp_size = 1 paired with arbitrary attention_cp_size to independently tune MoE and attention parallelism
- **added** — FlashInferCuteDslMoE layer for the standard FP4 MoE path
- **added** — Support for Qwen3-ASR with chunk-based streaming
- **added** — Support for Voxtral (Mistral speech-to-text)
- **added** — Support for Parakeet (NVIDIA Nemotron encoder)
- **added** — Support for Moss-VL model
- **added** — Support for SequenceClassification model architecture that powers the Score API
- **added** — Support for Stable Diffusion 3 medium diffusion model
- **added** — Support for ERNIE-Image diffusion model
- **added** — Support for JoyAI-Image-Edit diffusion model
- **added** — Incremental transfer support for Mooncake transfer engine in PD disaggregation
- **fixed** — Bug fixes for Eagle3/DFLASH aux hidden state capture during CUDA graph initialization, IntraNode NVLink, MTP-layer KV transfer, and disagg-prefill DP rank resolution

#### Highlights

- **CUDA 13 + Torch 2.11**: Default CUDA version moves to 13.0 across SGLang, sgl-kernel, and Docker images, and PyTorch is upgraded from 2.9 to 2.11 — modernizing the build matrix and unlocking newer kernels: #21247, #24162, #24183, #23593 ([tracking issue #21498](https://github.com/sgl-project/sglang/issues/21498))

- **Speculative Decoding V2 by default**: Spec V2 (with overlap scheduling to hide CPU overhead) is now the default, materially reducing per-step CPU cost for EAGLE/MTP/DFLASH paths: #21062

- **Decode Radix Cache for PD Disaggregation**: Decode-side prefix caching now works under prefill/decode disaggregation, recovering radix-cache hit rates and TTFT savings for long shared prefixes in disaggregated deployments: #19746

- **Day-0 / New Model Support**: Gemma 4, GLM-5.1, Qwen3.6, MiMo-V2.5 / V2.5-Pro, Ling-2.6-Flash, Mistral Medium 3.5, and Kimi-K2.6 — with cookbook recipes for tuned deployment commands. See [docs.sglang.io/cookbook](https://docs.sglang.io/cookbook): #21952, #23808, #23811, #23851, #23947, #23486, #23394

- **DFLASH Speculative Decoding**: New high-throughput spec-decode kernel from the kernel community, expanded across model backends and AMD ROCm: #22077, #22358, #22342, #23553

- **FA3 Kernels from the Kernel Community**: Drop-in FA3 kernels contributed by the community, integrated alongside FA4 to give users a high-performance option that's easy to maintain: #20796

- **LoRA support for DeepSeek-V3 and Kimi-K2**: LoRA now works on the largest MLA-based MoE models, including DeepSeek-V3 MLA LoRA and Kimi K2 — enabling adapter-based fine-tuning of frontier-scale models: #22323, #22381

- **Context Parallel (CP) Enhancements**: All-reduce + RMSNorm fusion under CP for end-to-end speedups, plus support for `moe_dp_size = 1` paired with arbitrary `attention_cp_size` so MoE and attention parallelism can be tuned independently: #21249, #22003

- **FlashInfer CuteDSL MoE Runner Backend**: New dedicated `FlashInferCuteDslMoE` layer for the standard FP4 MoE path, giving an additional high-performance fused-MoE option: #21339

##### New Model Support

Entries with a published cookbook recipe come first; entries whose cookbook page is still pending are grouped at the bottom.

* Gemma 4: #21952 (and follow-ups #22079, #24048, #22842; see [cookbook](https://docs.sglang.io/cookbook/autoregressive/Google/Gemma4))
* GLM-5.1: #22543, #23037 (see [cookbook](https://cookbook.sglang.io/autoregressive/GLM/GLM-5.1))
* Qwen3.6: #23486 (see [cookbook](https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.6))
* MiMo-V2.5 / MiMo-V2.5-Pro: #23808, #23811, #23851, #23945, #24118 (see [cookbook](https://docs.sglang.io/cookbook/autoregressive/Xiaomi/MiMo-V2.5))
* Ling-2.6-Flash: #23947 (see [cookbook](https://docs.sglang.io/cookbook/autoregressive/InclusionAI/Ling-2.6))
* Mistral Medium 3.5: see [cookbook](https://docs.sglang.io/cookbook/autoregressive/Mistral/Mistral-Medium-3.5)
* Kimi-K2.6: #23394, #23408 (see [cookbook](https://docs.sglang.io/cookbook/autoregressive/Moonshotai/Kimi-K2.6))
* Hunyuan v3 (Tencent, preview): #23533 (see [cookbook](https://docs.sglang.io/cookbook/autoregressive/Tencent/Hunyuan3-Preview))
* FLUX.1-dev ModelOpt NVFP4 (Diffusion): #22672 (see [FLUX cookbook](https://docs.sglang.io/cookbook/diffusion/FLUX/FLUX))
* FLUX.2-small-decoder (Diffusion): #22414 (see [FLUX cookbook](https://docs.sglang.io/cookbook/diffusion/FLUX/FLUX))
* Qwen Image ModelOpt FP8 (Diffusion): #23155 (see [Qwen-Image cookbook](https://docs.sglang.io/cookbook/diffusion/Qwen-Image/Qwen-Image))
* LTX-2.3 / LTX-2.3 two-stage / TI2V (Diffusion): #22182, #22667, #22869 (see [LTX cookbook](https://docs.sglang.io/cookbook/diffusion/LTX/LTX))
* Qwen3-ASR (chunk-based streaming): #22073, #22089
* Voxtral (Mistral speech-to-text): #21635
* Parakeet (NVIDIA Nemotron encoder): #23568
* Moss-VL: #23454
* SequenceClassification model architecture (powers the Score API): #22118

_[Truncated at 4000 characters — full notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.11]_

### v0.5.10.post1

- Date: 2026-04-09
- Version: v0.5.10.post1
- Original notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.10.post1
- Permalink: https://whatsnew.fyi/product/sglang/releases/v0.5.10.post1

- **fixed** — Resolve an issue in flashinfer's jit cubin downloader by bumping flashinfer from v0.6.7.post2 to v0.6.7.post3

**Full Changelog**: https://github.com/sgl-project/sglang/compare/v0.5.10...v0.5.10.post1

Bumps flashinfer from v0.6.7.post2 to v0.6.7.post3 to resolve an issue in its jit cubin downloader.