SGLang

Highlights

582 PRs from 194 contributors.

Kimi K3 day-0 support: A 2.8T-parameter multimodal LatentMoE (896 experts, top-16, routed in a 3584-dim latent space) with a 1M-token context, 69 KDA linear-attention layers interleaved with 24 MLA layers, and a MoonViT3d vision tower, shipping as a native MXFP4 checkpoint. SGLang serves it from day 0 with DCP, DSpark speculative decoding, chunked-prefill PP with TP decode, KDA-aware prefix caching, HiCache L2 over DCP, LoRA on the quantized weights, and reasoning, tool-call and OpenAI-compatible serving, verified on NVIDIA GB300 and AMD MI35x (#32541, #32828, #32890, #33025, #33112, blog, cookbook, roadmap).

MiniMax-H3 day-0 support: MiniMax's video generation model that produces a video and a synchronized stereo audio track in one request, served natively on SGLang-Diffusion across all three public task profiles: text-to-video-and-audio (t2va), first/last-frame conditioning (fl2va), and image/video/audio reference conditioning (ref2va, which also covers video-to-video). Verified on B200 (TP2 + Ulysses4), H100 (TP2 + Ulysses2), AMD MI300X and MI355X (Ulysses1/2/4/8), and 2x RTX 5090 with layerwise offload (#33275, cookbook).

Other new models added: EmbeddingGemma and LFM2.5 embedding models, nvidia/MiniMax-M3-NVFP4, plus cookbook recipes for Poolside's Laguna-S-2.1 family and Inkling-Small.

Initial support for the Rust frontend: Migrates the front half of the server, everything from network ingress up to the point a tokenized request is handed to the GPU scheduler, from Python to a multi-threaded Rust implementation (#29799).

DCP communication backends and q-replicate (Helix): The DeepSeek-MLA decode context-parallel path gains pluggable comm backends. a2a exchanges packed attention output plus fp32 LSE in a single NCCL collective per layer, with fp8 KV carried as uint8 byte transport; fi_a2a delegates the cross-rank exchange to the FlashInfer MNNVL kernel on GB200. --dcp-replicate-q-proj projects full-head Q locally and skips the per-layer Q head-dim all-gather. Select with --dcp-comm-backend {ag_rs, a2a, fi_a2a} (#21637).

DWDP for MoE prefill: A new prefill parallelism strategy that prefetches peer expert weights over NVLink P2P and computes all experts locally, removing EP all-to-all token dispatch. On 4x B200 with gpt-oss-120b, prefill-only, DWDP4 reaches 1.92x over DEP4 at MNT 32K / ISL 32K, and 506K vs 329K tok/s (1.54x) at saturation (CONC=128, ISL=8K). Enable with --dwdp-size; the authors mark it early-development (#29778).

Session-reference-aware Unified Radix Cache: For agentic and RL-rollout workloads, requests can carry a stable session_id so eviction knows which prefixes an active session still references, instead of evicting purely by cache policy. Release the references with /close_session. Opt in with --enable-session-radix-cache (#29173).

SM90 FP8 MegaMoE for DeepSeek-V4: Adds the DeepGEMM MegaMoE A2A path on SM90 for DeepSeek-V4-Flash/Pro FP8, including the pre-dispatch JIT kernel and FP8 expert weight preparation. Guarded behind SGLANG_OPT_USE_DEEPGEMM_MEGA_MOE=1 (#29016).

Faster large-MoE model loading: Oversized or non-contiguous CPU weight views were driving pathological H2D transfers, with DeepSeek-V4-Pro TP8 spending 27 to 32 minutes in H2D on some ranks. Copying those views into contiguous storage before H2D cuts full model loading from about 35 minutes to 6m20s (5.6x), and GPT-OSS-20B BF16 from 545s to 70s (7.8x), with Qwen3.5-397B measured at 1.93x to 2.3x. Opt in with SGLANG_MOE_COPY_WEIGHT_VIEWS_BEFORE_H2D, off by default (#32315).

Lower DeepSeek-V4 memory on AMD: Removing unnecessary expert padding drops MI355X FP4 MoE model weights from 159.07 GB to 112.36 GB, and bringing the HIP compress-state pool into the memory_saver KV_CACHE region lets colocated RL reclaim it, cutting the measured training-phase footprint from about 143 GiB to 87 GiB per GPU (#31450, #31747).

Faster engine recovery: Large-model restarts cost 3 to 6+ minutes today, about 6.5 minutes for Qwen3-235B FP8 on 4 GPUs, because weights reload from storage and CUDA graphs recapture. A weight-cache daemon holds weights per GPU so a restarting engine can recover from cache instead (#27139).

Lower host overhead in hybrid-linear MTP decode: Under spec-v2 overlap scheduling each decode step runs draft, verify and extend CUDA graphs, and the eager seams between them become GPU idle time at low concurrency. This trims that host work so the host stays off the critical path (#32219).

Dependencies: flashinfer 0.6.15.post1 (#31927), sgl-deep-gemm 0.1.5.post1 (#32345, #33143), helion 1.4 (#32562), mooncake 0.3.12.post1 (#32302), dynamo-tokenizers 1.7.0 (#32981). PyTorch stays at 2.11.0 and the CUDA base image at 13.0.1.

Full release notes by category below; breaking changes and known issues are at the end.

New Model Support

Model	Type	PRs	Cookbook
Kimi K3	Autoregressive (Multimodal)	#32541, #32828, #32890, #33025	link
MiniMax-H3	Diffusion	#33275	link
MiniMax-M3-NVFP4	Autoregressive	#31989
EmbeddingGemma	Autoregressive (Embedding)	#32375, #32383	link
LFM2.5	Autoregressive (Embedding)	#28691	link

Kimi K3

• [Kimi] Support kimi-k3: #32541 ⭐
• [Kimi] Support DCP + DSpark (ported from kimi-k3 branch): #32828
• [Kimi K3] Add reasoning, tool-call, and OpenAI serving support: #33025
• feat(kernels): port standalone Kimi K3 kernels: #32890
• [Feat] DCP + HiCache L2 Support (ported from kimi-k3): #33112
• Replace Kimi K3 DeepGEMM patch with 0.1.5.post1: #33143
• docker: add Kimi K3 images: #32760

Rust Server

A native Rust serving layer: tokenizer manager, ingress validation and egress, an OpenAI-compatible API server, and PD disaggregation support, shipped as prebuilt release artifacts.

• support rust sglang server: #29799
• create rust workspace: #32014
• init sglang rust server project: #32256
• add the rust server tokenizer, detokenizer, and egress modules: #32872
• add the rust server ingress request validation and api server common types: #32873
• add the rust server ingress tests, guard, and submit modules: #32874
• add the rust server api frame codec and http server entry: #32875
• add the rust server native api handlers and runtime threads: #32876
• wire the rust server modules into lib, runtime, and tokenizer manager: #32877
• sglang rust server tokenizer manager, ring and runtime: #32358
• feat: rust sglang server openai apis: #33103
• [rust-server] PD disaggregation support: #33125
• [rust-server] Reland: fix TCP-layer TTFT stalls (#33026): #33269
• rust server build release artifacts: #33096

Speculative Decoding

• [MTP] Cut spec-v2 host-seam overhead in hybrid-linear MTP decode: #32219 ⭐
• [KDA] Add target_verify support for speculative decoding: #26888
• [DSPARK] Grammar-constrained decoding, incl. tool_choice=auto: #31753
• [DFLASH] Support grammar-constrained decoding in speculative verify: #30096
• [Spec] Enable grammar overlap scheduling for STANDALONE speculative decoding: #32110
• Overlap grammar (constrained decoding) with speculative decode verify: #31488
• [Spec] Support sampling in the DSPARK graph-folded draft proposal: #33298
• [Spec] Add trtllm_mha support for Gemma 4 MTP draft attention backend: #25545
• [Perf] Fast-path chain-style draft token organization in multi-layer EAGLE: #32887
• [Perf] Skip the target-verify tree mask fill when the backend never reads it: #32886
• [Spec] Compact the target-verify mask when nothing reads it: #32920
• [Perf] Fold dspark dense draft embedding into the draft graph via forward_embed: #31985
• [Perf] Stack dspark dense draft per-layer ctx KV projection into one GEMM: #31986
• [Mamba] Support speculative decoding with extra_buffer_lazy: #30437
• [gdn] fused replayssm ring write into flashinfer gdn mtp verify kernel: #33102
• [gdn] support replayssm with extra buffer: #32692
• [6/6][kimi-deterministic] Use deterministic seeded coins for EAGLE rejection sampling: #30822
• [Speculative Decoding] Fix GPT-OSS EAGLE3 hidden states: #32334
• [EAGLE] Handle NaNs in fused top-k=1: #32396
• Pack aux hidden states into a preallocated buffer: #28956
• Support SGLANG_SIMULATE_ACC_LEN for DFLASH: #32595
• Fix stop boundaries for grammar-constrained speculative decoding: #31738
• Fix DSpark loading for hybrid DSV4 NVFP4: #33276
• Fix DSpark and DP/EP: #33098
• [Fix] Clamp degenerate all-sentinel draft rows to token 0 in dspark _online_combine_kernel: #32277
• [Fix] Bound FULL_MASK verify-mask reuse by the captured max_bs: #33127

Piecewise & Breakable CUDA Graph

• Turn on breakable prefill cuda graph for dp attention by default: #31682
• [FullCG] Support chunked cached-prefix prefill: #30825
• [FullCG] Preserve attention LSE through the custom-op boundary: #31050
• [BCG][3/N] Enable bcg on dsa & deepep a2a backend: #31987
• [BCG][4/N] Enable bcg on megamoe & flashinfer a2a backend: #33150
• [CUDA Graph] Allow custom decode graph runners: #33553
• [cuda_graph] Gate breakable-CG capture_inputs retention to DP-gather paths: #32678
• Enable multimodal prefill BCG for VL and audio models: #30872
• Enable post-capture KV sizing with DP attention: #33427
• fix(server): capture legal multi-request prefill CUDA graph batches: #30206
• Fix BCG circular import during server startup: #33371

Attention Backends

• [Attention Backend] Add HPC-Ops attention backend: #30540 (blog)
• [Attention Backend] Extend hpc_ops dynamic-scheduled decode to bf16: #32304
• [DSA] Q8KV8 FP8 Sparse Prefill on GLM-5.2 & DeepSeek-V3.2: Q8-Path & Shared-Path Optimizations: #31888
• [unified-memory] Support MLA-hybrid-Mamba (Kimi-Linear) on the Triton backend: #32971
• [unified-memory] Let Kimi-Linear use the paged MLA attention backends: #32972
• [unified-memory] Support fa3, the default MLA backend on pre-Blackwell hosts: #33046
• feat(inkling): migrate short convs onto the ShortConv attention backend: #33023
• [GDN] Support FlashInfer GDN prefill with extra-buffer radix cache: #29735
• fix(gdn): skip the -1 padding sentinel in the chunked extend kernel: #33810
• [Kernel] Fuse KV-cache writes for asymmetric K/V (head_dim != v_head_dim): #32813
• [Perf][DSA] Pass topk_length to flash_mla_sparse_fwd in the sparse attention path: #31128
• [Perf] Skip page-table columns past kv length in DSA draft-extend metadata kernel: #31981
• [Perf] Skip blocks past per-request live length in full-width Triton kernels: #32109
• Support a same-size mixed q dtype in the fused RoPE kernels: #31834
• Avoid TRTLLM prefill output copy: #33306
• Fix attention backends for models with per-layer head counts (num_attention_heads_per_layer): #32625
• [Fix] Route asymmetric-KV models to fa4 on SM100 and pin MiMoV2 FP8 MoE to flashinfer_trtllm: #32818
• [Fix] Fix trtllm_mla backend + fp8 kv cache without rope: #32181
• [Fix] Support ENCODER_ONLY target-verify in the trtllm_mha backend: #32178
• [Fix] missing max_context_len on HybridAttnBackend: #32690
• fix(dsa): correct packed FlashInfer top-k and backend selection semantics: #32490
• fix(dsa): fail fast on fp8_e4m3 KV with tilelang DSA backend on CUDA: #31346
• [KDA] Fix mixed exponent bases in Triton chunk prefill: #31904
• [FIX] Prevent Lightning Attention extra-buffer mamba state corruption: #29973
• Add configurable FlashInfer autotune skips: #31389
• Fix FlashInfer MNNVL workspace size check: #32318

MoE & Expert Parallelism

• [Feature] Add DWDP (Distributed Weight Data Parallelism) for MoE prefill: #29778 ⭐
• Add SM90 FP8 MegaMoE support for DeepSeek-V4: #29016 ⭐
• [MoE Backend] Add HPC-Ops FP8 MoE runner backend: #30541 (blog)
• Integrate pplx a2a backend: #30756
• Support DeepGEMM for standard MoE dispatch: #33128
• [DSV4] Support megamoe for CP: #29569
• [MoE] Make DeepEP auto serve flashinfer_cutedsl FP4 (coerce to low_latency) + guard: #29523
• [2/N] elastic-ep: Enable EPLB after scale-up: #30553
• [Elastic EP] Centralize Mooncake PG configuration: #31708
• [Elastic EP] Fix recovery lifecycle and add manual coverage: #31744
• perf: speed up marlin moe with occupancy-aware launch specialization: #31552
• Migrate CompressedTensorsW4A4Nvfp4MoE TRT-LLM path onto MoeRunner: #32248
• Load initial expert location metadata on CPU: #32435
• Fix silently wrong EPLB output with --moe-a2a-backend none (rank-invariant dispatch): #32962
• Fix MoE reduce-scatterv eligibility check: #32663
• Fix pad-row top-k masking with custom_routing_function under DP attention: #31838
• Add DeepSeek-reference 1e-20 epsilon to top-k renormalization to prevent 0/0 NaN: #31017
• [Fix] Clear stale FlashInfer BF16 MoE index cache: #33016
• Optimize LongCat-Flash router GEMM with the HPC-Ops bf16xfp32 kernel: #30247
• [Fix] Reject online weight updates while the HPC-Ops router GEMM split cache is active: #31943

Quantization

• Support CuteDSL GEMM BF16 on SM100 on by default when allowed by heuristic: #30567
• [Quant] Support NVFP4_AWQ checkpoints in ModelOpt FP4 path: #31825
• [Quant] Keep the flashinfer_deepgemm FP8 GEMM to 1 <= M < 32: #32843
• Qwen3.5-MoE: support modelopt_fp4 checkpoints that quantize attention (+ load baked FP8 KV scales): #31220
• [Qwen3.5][MTP] Support FlashInfer CuTe DSL for online NVFP4 draft MoE: #31382
• Fixing MXFP8 online quantization pipeline: #31510
• [Fix] Restore online MXFP8 quantization for linear layers: #32953
• [Bugfix] compressed-tensors: mixed-precision checkpoints silently load unquantized: #32736
• Fix compressed-tensors NVFP4 MoE W13 layout: #32430
• fix(marlin_nvfp4): only apply routed_scaling_factor in moe_sum_reduce: #31762
• [Perf] Halve the non-finite sanitization overhead in per_token_group_quant: #32296
• [JIT] Trait-driven per_token_group_quant: unify the quant kernel family (flat + masked): #30924
• [JIT] Restore the previous division behavior in per-token group quantization: #32616
• Fix FP8 Triton dtype selection on A100: #31340

Parallelism & Disaggregation

• [Feature] DCP: A2A + FlashInfer-MNNVL comm backends and q-replicate (Helix): #21637 ⭐
• [DeepSeek V4] CP decode opt: slice repeat attention weights to local TP partition: #27657 ⭐
• [4/N][CP] Support interleave strategy for cp v2: #30482
• [CP] Fuse zigzag attention into a single call: #33137
• [CP] Support breakable CUDA graphs for zigzag strategy: #33136
• [CP]: FIx some issue for glm5.2 cp v2: #33100
• Support DCP for Kimi Linear model: #32612
• feat: support Kimi Linear PD disaggregation with DCP: #32837
• [Bugfix] Fix Kimi-Linear state transfer across heterogeneous TP: #32262
• [DCP] Match the replicated draft KV pool's page granularity to its allocator: #33348
• [DCP] Bound a request by the aggregate KV pool, not one rank's share: #33448
• [Fix] Honor FlashMLA natural-log LSE in DCP reduction: #33065
• fix(mem_cache): state the MLA KV bound in the DCP index space: #33432
• [comm] Enable multi-node custom-AR v2 on a single NVLink clique: #32339
• [Fix] Enable graph capture and MSCCL++ for attention TP groups: #31629
• Disable extra NCCL CUDA event synchronization with symm mem: #27089
• [PD] NIXL connector: shard by destination: #32025
• [PD] pool decode bootstrap HTTP sessions: #31543
• [PD] Handle abort requests in PP mode: #32797
• [PD+PP] Honor PP consensus for bootstrap and prealloc: #31869
• [PD] Prevent decode scheduler from blocking on ZMQ sends to a stalled prefill peer: #31144
• [PD] Fix false health-503 during decode retraction re-admission: #33118
• [PD] Drain NIXL completion notifications before enforcing the WaitingForInput timeout: #32267
• [Disagg][NIXL] Fix heterogeneous attn-TP KV transfer for replicated GQA heads (NIXL_ERR_NOT_FOUND): #31968
• [Disagg][StagingBuffer][1/2] Robustness and failure handling: #31217
• fix(disagg): support pipeline-parallel hybrid-linear transfer: #32270
• [BugFix][EPD] Harden zmq_to_scheduler receiver failures; sync error info across TP: #31592
• [EPD] Fix HTTP dispatch lock blocking cross-request encoder batching: #31275
• [EPD] Make encoder register/unregister health-check robust: #31576
• [BugFix][EPD] Fix Mooncake source-MR lifecycle for multi-TP /send: #32071
• [BugFix][EPD] Early-release mooncake GPU embeddings; fix gpu_id via scheduler.ps: #31591
• [HiSparse]Fix DeepSeek V4 HiSparse PD Transfers with Separate Host and Device KV Indices: #31901
• support dp attn with client lb: #33105
• Fix token count localization for replicated attention-TP forwards: #32411

Scheduler & Runtime

• [FEAT] Support fast engine recovery through weight cache: #27139 ⭐
• feat(grpc): add generation request semantics: #32588
• [Scheduler] Enable decode retraction ordering under speculative decoding: #32023
• Allow optimistic prefill with L2 hierarchical cache and write-back policy: #33545
• Bound prefill delayer all-branch delay and decay the max_prefill_bs high-watermark: #32880
• Negotiate PrefillDelayer only after KV-budget admission checks: #31835
• [Bugfix] Fix prefill suspension caused by delayed negotiate_should_allow_prefill invocation: #32389
• Fix SWA admission livelock on cached-prefix resumes: #32379
• [core/loader] Add presharded load format: #24256
• Support fastsafetensors no-GDS loading and page-cache release: #31859
• Fix async loading of RunAI-streamed tensors: #32896
• Fix DeepSeek V4 loading with RunAI Model Streamer.: #30240
• [Fix] Fail fast when a safetensors index references missing shard files: #32279
• AutoWeightLoader support Sglang native models 1: demo: #28671
• Fix reward/classification models broken by load_weights v2 dispatch (#28671): #31988
• perf(startup): skip unused PyTorch headers for KV VMM allocator stub: #33126
• Reduce startup log noise and fix Dynamo / CUDA-graph edge cases: #33428
• [Feature] Add leveled invariant-check primitive for nan/inf/oob validity checks: #32308
• [Observability] Add startup, memory, and hybrid SWA diagnostics: #33375
• [metrics] Split tokenizer request metrics by stream: #32734
• Add stream label to TTFT metrics: #32363
• fix(metrics): clear forward occupancy on idle: #33562
• Report accelerator type in /v1/loads: #32348
• [Router] Report accelerator count in /v1/loads: #33548
• Add prefill and decode load counters to LoadSnapshot: #32245
• [PD] Add a queues.prealloc_ready counter to the load snapshot: #33133
• [Fix] Include disagg prefill waiting queue in FPM: #32122
• fix: prevent ReqTimeStats from being dropped during IPC serialization: #31339
• config: make ServerArgs read-only with a single audited mutation entry: #31811
• runtime_context: per-role namespace enforcement behind SGLANG_ROLE_NAMESPACES: #33172
• [misc] Deep-merge nested config overrides and parse request bodies with orjson: #33351
• [Perf] Free KV pages by segment in the paged allocator without a device sync: #32701
• [mem_cache] Build empty-prefix last_loc sentinel on-device to avoid per-call H2D sync: #32575
• [Kernel] Skip KV writes to reserved padding slots: #32477
• Skip dist_init/nccl port prechecks when the dist init method is overridden: #31410
• [BugFix] Prevent TBO crash when return_logprob is enabled: #32180
• [BugFix] Fix DS/Kimi crash on non-first PP ranks when resolving input length: #31752
• [Fix] Two root causes of the H100 deepep TBO CI break: scale-tensor use-after-free + missing non-finite quant sanitization: #32188

HiCache & Radix Cache

• feat: Session-reference-aware Unified Radix Cache for agentic multi-turn workloads: #29173 ⭐
• Radix Cache Split: Spin off TreeCore: #29901
• [Hicache][1/2]Support Mamba branching in Unified Radix Cache with HiCache: #31181
• feat(hicache): Add shared memory allocator for host KV cache: #29326
• [HiCache] Add model-aware key isolation to Mooncake Store: #31920
• Add Mooncake tenant id support: #30256
• Eliminate redundant DSA state transfers (Mooncake): #32620
• [HiCache] Merge HiCache event checks to reduce decode overhead: #30511
• [HiCache]: Optimize hybrid/DSA L3 prefetch result sync and usable-prefix clamping: #31443
• fix(hicache): support staged write-back for asymmetric MHA: #30981
• Write-back policy fix for unified tree: #31845
• Fix --hicache-size allocating ~2x host memory on hybrid SWA: #32373
• Fix --hicache-size allocating ~2x host memory on hybrid Mamba: #32915
• [UnifiedTree] fix: drop prefetched host refill under an un-backed-up parent: #31902
• [UnifiedRadixCache][mamba] Fix mamba state corruption and slot leak when load_back aborts: #30986
• [Fix] Evict only the KV shortfall in evict_from_tree_cache: #32016
• [Fix] Unify pinned host pool release on graceful shutdown: #32029
• fix(hisparse): correct DSA KV memory budget: #31992
• perf(hisparse): eliminate redundant swap output fill: #32483
• [Mamba] Add a per-path cap for cached states: #31230
• Skip mamba lock during decoding: #32228
• [Fix] Reserve the mamba pool's +1 padding slot in the memory budget solve: #32184
• Fix unnecessary gather/scatter on CPU for non-contiguous Mamba statepool: #31754
• [XPU] Add XPU device support for LMCache radix cache integration: #23534

LoRA

• [LoRA] Support LoRA under the breakable/full prefill CUDA graph: #30988
• Split #32584 into 2/2: [LoRA] Shard attention LoRA by attn-TP and allow dynamic LoRA with dp attention: #32708
• Split #32584 into 1/2: [LoRA] Guard DP-attention idle forwards against stale LoRA batch state: #32707
• [LoRA] 1/n Per-rank tensor serialization for load_lora_adapter_from_tensors under dp_size > 1: #32580
• [LoRA] Guard TMA down path for LoRA hooks: #31608
• [lora] Fix WAR race: never write MoE runner output into hidden_states in place: #31870
• [LoRA] Fix Marlin MoE kernel import: #32884
• LoRA: Ascend: Update ascend LoRA backend to support new kernels: #15912

Multimodal

• [mm] sglang-mm: server vision pipeline core (fetch/driver/pipeline) + Qwen VL: #32364
• vlm: parallelize multimodal preprocessing with customized worker num: #31438
• [Perf] Broadcast single-image DP vision embedding instead of pad-to-max all-gather: #33307
• [Perf] Speed up the Kimi-K2.5 vision path and match PIL bicubic in the GPU resize: #33349
• [EPD][VLM] Fix Kimi-VL 2D encoder grids: #32104
• [mm] Accept per-item embedding lists from DataEmbeddingFunc: #31826
• [mm] Handle per-item embeddings in cache misses: #32498
• Decode input_audio media containers with PyAV & Update memory profiler: #31832
• fix(vlm): materialize Qwen3-VL features on the vision device: #31596
• vlm: reject moss vision metadata mismatches: #31957
• [Bugfix] Place empty Qwen encoder-DP embeddings on the communication device: #31663
• [Fix] --mm-process-config crash when video config contains: #30260
• [Fix] Drop deprecated multimodal processor residency state: #33308

Model Support & Optimizations

• [Feature] Support return_hidden_states="last": #30177
• Add return_token_ids support to completions and chat completions APIs: #30917
• embedding: centralize capabilities and complete OpenAI compatibility: #32481
• [Model] Support standalone text-only Qwen3.5 checkpoints: #32401
• model: serve bare Qwen3Model backbone natively as an embedding model: #32457
• [feat] Opt-in flat response format for prompt top logprobs: #32078
• [feat] Optional base64 encoding for the flat prompt top logprob arrays: #31960
• [perf] Assemble flat prompt top logprobs scheduler-side as numpy arrays: #32223
• Use native batched llguidance mask generation: #32412
• [Spec] Hold the grammar bitmask in one GrammarMask type across all decode paths: #32409
• [Spec] Consolidate the grammar sync decision into ScheduleBatch.grammar_needs_sync: #32353
• [Spec] Share the grammar mask build and verify-tree staging across spec workers: #32393
• [Spec] Derive NGRAM grammar tree links on the host instead of reading back retrive_next_token: #32380
• support regex that compatible with python re lib however apply more l...: #32676
• fix(sampling): reject conflicting structural tag constraints: #32525
• [Fix] Treat an empty grammar constraint as unset in SamplingParams: #33328
• Add 'anyOf' schema support for qwen3_coder tool call parser: #30832
• Fix dropped tool calls when a stream delta carries several: #31860
• Fix Inkling tool-call parsing recovery, content handling, and streaming: #32861
• Fix LFM 2 tool parser.: #27614
• [Fix]: render tool_reference schema regardless of tool_result part order: #32522
• Treat partial_json_parser AssertionError as incomplete JSON: #31975
• Flush dropped reasoning at stream end when stream_reasoning=False: #32225
• fix(reasoning): let --enable-strict-thinking works for DeepSeek-V4: #32400
• fix(reasoning): honor Poolside template thinking defaults: #32540
• Guard min_new_tokens penalizer against None eos_token_id: #31973
• [Inkling] Add minimal DFLASH support: #31840
• [Inkling] Hold the short-conv per-step state on one metadata struct: #33116
• Fix broken Nemotron DP attention: #33123
• [Nemotron] Fix decode track-save reading the stale tail of the CUDA-graph track buffer: #32555
• [Nemotron] Hoist mamba track-mask host syncs out of the per-layer prefill path: #32589
• [LFM2] Wire Lfm2MoeForCausalLM into the LFM2 serving override tables: #30780
• Fix LongCat n-gram token-table crashes on padded batches: #31312
• fix(qwen3.5): restrict MoE weights to local PP layers: #32022
• [minimax-m3] fp8 attention GEMMs on SM100 (fp8_e4m3 KV + trtllm_mha): #30971
• Prewarm DSV4 MHC post kernel at model load: #30741
• fix mqa preshuffle layout issue for deepseek v4: #31563
• [DeepSeek-V4] Fix nvcc 13 crash building the topk_v2 kernel: #32910
• [RL] DSV4: dispatch indexer topk_transform_512 through DSATopKBackend: #31087
• [RL] DSV4: add env to quantize SWA KV cache from bf16-rounded values: #31086
• [RL] Support FlashInfer TRT-LLM NVFP4 MoE in the RL weight checker: #31085
• [DLLM] vectorized joint/low-confidence decoding and skip redundant attn init: #21094
• [Bugfix] Fix Cohere2MoeConfig import crash from huggingface_hub @strict: #31769

Kernel Library (sglang.kernels, RFC #29630)

• [Kernel] RFC #29630 finale: retire sglang.jit_kernel into sglang.kernels: #32072
• [Kernel] Move sgl-kernel under sglang.kernels.aot: #32648
• [Kernel] Phase 3+4: move JIT infra + operator groups into sglang.kernels (RFC #29630): #31666
• [Kernel] Phase 4 batch-2: migrate JIT operator groups into kernels.ops (no shims) (RFC #29630): #32015
• [Kernel] Phase 4 batch-3: migrate tangled JIT subsystems + new groups into kernels.ops (RFC #29630): #32045
• [Kernel] Sweep missed dedicated kernels into kernels.ops (moe/quant siblings + dspark) (RFC #29630): #32160
• [Kernel] Classification cleanup: unify jit naming, drop empty/model groups, add elementwise (RFC #29630): #32148
• [Kernel] Reclassify kernel tests by ops group + move helpers out of the package (RFC #29630): #32128
• Delete sgl-kernel AOT bmm_fp8, use flashinfer.bmm_fp8: #31202
• Delete sgl-kernel AOT router GEMM and fused A GEMM: #30280

SGLang-Diffusion

• [diffusion] model: support minimax-h3: #33275
• [diffusion] FLUX.2 VAE decoder fast path behind quality=high (H200: 1024^2 97.6->29.2 ms, 2048^2 437.2->168.5 ms): #33451
• [diffusion] Fuse DiT FFN tanh-GELU into up-proj GEMM (cublasLt epilogue) behind quality=high (Qwen-Image 1024^2 denoise 12.36 -> 12.05 s on H200): #33536
• [diffusion] feat: add dynamic cuDNN SDPA attention backend: #30090
• [diffusion] feat: support resident layers for DiT: #31538
• [diffusion] feat: unify encoder folding and batch data-parallel encoding: #30211
• feat(diffusion): add regional torch compile: #32696
• perf(diffusion): decode Wan VAE in BF16: #32697
• [diffusion] optimization: accelerate CUDA video output finalization: #32784
• [diffusion] optimization: support cuda-ipc zero-staging all-to-all for 2-rank Ulysses: #31854
• [Diffusion] offload rollout weights to pinned host memory: #32032
• [diffusion] fix: preserve tensor stride when offloading rollout weights to pinned host memory: #32420
• [Diffusion] Return scheduler sigmas snapshot in rollout dit_trajectory: #32683
• [diffusion] fix: reject ring parallelism where it would silently miscompute: #33353
• fix(diffusion): size VSA top-k from padded blocks: #32695
• [diffusion] fix: keep fused qk-norm-rope out of dynamo tracing: #31849
• [diffusion] fix: fix dual-DiT models crash with (1,)-placeholder weights after compile-time offload: #32743
• [diffusion] fix: per-shard FP8 scale shape for single-GPU fused linears: #32157
• [diffusion] fix: fix local-path detection for MiniMax-H3 and other non-diffusers models: #33365
• [diffusion] fix: don't self-kill diffusion worker when PID 1 is the real parent: #31361
• [Diffusion] Fix component accuracy topology reuse: #33317
• [vla] fix: pi05 models does not apply scale factor for language embeddings: #33367

Local & Desktop AI

• feat: SM120 (Blackwell Desktop) support for GLM-5.1 inference: #26928
• Add FP4 Indexer for DeepSeek V4 on SM120: #27059
• Enable GPT-OSS FlashInfer MXFP4 on SM120: #32668
• Change the FP8 per-tensor GEMM backend on SM120 to cuBLAS: #31961
• [SM120] Only split touched SWA pages in FlashMLA page-split kernel: #32320
• [DSV4/SM120] Allow fused MHC opt-in with standalone TileLang pre disabled: #30954
• Sm120 scatter fallback: #31669
• [MLX] Fix overlap-loop request bookkeeping and graceful shutdown: #32447
• [MLX] Size request capacity by attention DP: #32115
• [MLX] Handle configs without quant_method in Humming: #31460
• [diffusion] fix: fix diffusion output stability on mps: #30017

New recipes: RTX 5090 DeepSeek-V4, DGX Spark for Inkling-Small, MiniMax-H3 on 2x RTX 5090 (TP2 + layerwise offload).

AMD / ROCm

• [AMD] Add fused all-reduce RMSNorm per-group quant for Qwen3.5 FP8: #24651
• [AMD][MXFP4] Reland "Online MXFP4 quantization 2/N - FP8 to MXFP4 requantization on AMD GPUs": #28291
• [AMD] Cache AITER expert mask across decode: #31889
• [AMD] Derive AITER verify tokens-per-req from input shape: #31221
• [AMD] Speed up DSV4 MoE weight loading from mmap views: #32315
• [AMD] Fix DeepSeek-V4 FP4 MoE expert memory bloat: #31450
• [AMD] DSv4: bring HIP compress-state pool into the memory_saver KV_CACHE region: #31747
• [AMD] Fix DeepSeek-V4 fused-RMS FP8 scale metadata on gfx950: #31727
• [AMD] [Fix] Enable aiter hd256 FP8 prefill FMHA on gfx950: #33399
• [AMD]Qwen3.5 integration gfx950 fmha fp8 hd256: #32046
• [Fix][AMD] Qwen3.5 MoE: disable global-slot shared-expert fusion under per-rank EP backends (MoRI + dp-attention init crash): #31793
• [AMD] MiniMax-M3: opt-in custom/quick all-reduce on ROCm: #32230
• [AMD] Minimax-M3 : unblock mxfp8 block convert on gfx950: #32036
• [AMD] Enable mamba JIT transfer kernel on ROCm (fix transfer_kv_mamba NameError): #31741
• [AMD] add Gemma3RMSNorm.forward_hip to unbreak ROCm: #32613
• [AMD][Fix] Restore aiter-padded MoE weight dims for serialized checkpoints: #33090
• [AMD] Fix JIT compile failure in sgl_kernel/warp.cuh: #33195

NPU / Ascend

• [NPU] Optimize DeepSeek-V4 performance: #31931
• :sparkles: [llm][npu][quant] Add W8A8 MXFP8 quantization for Qwen3 MoE on Ascend NPU: #30768
• [NPU] adapt dflash v2 on npu: #31739
• [NPU] Adapt MiMo-V2.5-W8A8: #29131
• [NPU] Enable automatic ascend_attn selection for vision attention and graph runners: #31948
• [NPU] ascend fuseep use moe ep group: #32040
• [NPU] Fix MTP IndexShare warm-up for attention DP and prefill CP: #32210
• [NPU] fix dsv4 mtp condition on NPU graph: #32711
• [NPU] [FIX] Fix performance degradation of Qwen3.5-397B-A17B: #32130
• [Bugfix] [NPU] Fix w4a8 MoE performance degradation: #32113
• [NPU] Fix LLaDA2 MoE OOM after the FRACTAL_NZ cast, re-enabling the NZ speedup: #31772
• [NPU] Determine the topk norm_type through scoring_func: #31393
• [NPU] Acc fix for afmoe model introduced by topk refactor.: #31280
• [NPU] bugfix for extra device memory on Ascend: #30112
• [NPU][Fix Issue]: Send expert weights contiguous tensor across cards during EPLB rebalance: #32001
• [NPU]Add Ascend transfer version compatibility.: #31189
• [NPU] Update non-vit vision part for cumulative seqlen: #31867
• :bug: [llm][npu][quant] Fix ModelSlim MXFP4 packed weight loading: #32013
• add fill_draft_extend_prepare_buffers_native for NPU: #32427
• sgl-kernel-npu tag update to 2026.7.27: #32596

CPU / Intel / XPU

• [Intel GPU] Add XPU Platform support: #31949
• [XPU] Enable breakable prefill CUDA graph on XPU: #30273
• [XPU][GDN] add XPU path for causal_conv1d_fn and causal_conv1d_update: #31250
• [Intel GPU] calculate free memory based on allocated memory for XPU: #32044
• [sgl-kernel][CPU] add kernel for shm_allgather_into_tensor and shm_reduce_scatter_tensor: #13397
• [CPU] refactor rope kernels: #31897
• [CPU] Fix mxfp4 padding size: #31334
• [Bugfix] Fix CUDA import on non-CUDA platforms: #31919

Dependencies

• Bump FlashInfer to 0.6.15.post1: #31927 ⭐
• Bump sgl-deep-gemm to 0.1.5: #32345
• Replace Kimi K3 DeepGEMM patch with 0.1.5.post1: #33143
• Bump helion version to 1.4: #32562
• chore: bump mooncake version to 0.3.12.post1: #32302
• bump dynamo-tokenizers to 1.7.0: #32981
• [AMD] ROCm AITER pin reverted to 9127c94, then updated to d9e5ef7: #32879, #32939
• Fix PyPI release: drop the git-only sgl-eval dep from packaged metadata: #32354

Breaking Changes & Upgrade Notes

• helion jumps 0.2.6 to 1.4, a major-version move for anyone depending on helion-backed kernels: #32562
• sglang.jit_kernel is retired into sglang.kernels, completing RFC #29630. Imports from the old module path must move: #32072, #31666, #32015, #32045
• Breakable prefill CUDA graph is now on by default for DP attention: #31682
• CuteDSL BF16 GEMM on SM100 is on by default when the heuristic allows it: #30567
• sgl-kernel AOT bmm_fp8 is deleted in favor of flashinfer.bmm_fp8; the AOT router GEMM and fused A GEMM are also removed: #31202, #30280
• The FP8 per-tensor GEMM backend on SM120 switches to cuBLAS: #31961
• Unfetchable or unparseable multimodal inputs now return HTTP 400 instead of 500, and text-only models reject media input outright: #31417, #32914
• Diffusion request-level quality is restricted to two validated tiers, lossless (default) and high: #33453
• ServerArgs is being made read-only behind namespace config bags. Code that mutated ServerArgs at runtime must route through the new accessors: #31809, #31811, #33334, #33338

Known Issues

• The FlashInfer rmsnorm + quant fusion for SM90/SM100/SM120 was landed and reverted this cycle (#32994, #33455). It is not in this release.
• Breakable CUDA graph is disabled for NemotronH pending a fix: #33130
• test_update_weights_from_disk is skipped on ROCm pending a reload fix (#31924): #31925
• The SM120 DeepGEMM release-pipeline test is temporarily skipped: #32193

Full Changelog: v0.5.16...v0.5.17

New Contributors

• @windscope made their first contribution in https://github.com/sgl-project/sglang/pull/31738
• @kfhfar made their first contribution in https://github.com/sgl-project/sglang/pull/31825
• @fkuner made their first contribution in https://github.com/sgl-project/sglang/pull/29973
• @ilyasher-harmonic made their first contribution in https://github.com/sgl-project/sglang/pull/30832
• @tedzhouhk made their first contribution in https://github.com/sgl-project/sglang/pull/32122
• @calvin0327 made their first contribution in https://github.com/sgl-project/sglang/pull/31460
• @libinta made their first contribution in https://github.com/sgl-project/sglang/pull/23534
• @houseroad made their first contribution in https://github.com/sgl-project/sglang/pull/31826
• @mosya415 made their first contribution in https://github.com/sgl-project/sglang/pull/31346
• @yangspirit made their first contribution in https://github.com/sgl-project/sglang/pull/29326
• @SovietPower made their first contribution in https://github.com/sgl-project/sglang/pull/31144
• @hsthe29 made their first contribution in https://github.com/sgl-project/sglang/pull/30096
• @yyq0210 made their first contribution in https://github.com/sgl-project/sglang/pull/26888
• @shanemort1982 made their first contribution in https://github.com/sgl-project/sglang/pull/31753
• @ormandj made their first contribution in https://github.com/sgl-project/sglang/pull/30954
• @daii-0818 made their first contribution in https://github.com/sgl-project/sglang/pull/31992
• @gongy made their first contribution in https://github.com/sgl-project/sglang/pull/31417
• @yilin-void made their first contribution in https://github.com/sgl-project/sglang/pull/26928
• @xiaolin2004 made their first contribution in https://github.com/sgl-project/sglang/pull/32448
• @xiaojun-zhang made their first contribution in https://github.com/sgl-project/sglang/pull/32104
• @waynel96 made their first contribution in https://github.com/sgl-project/sglang/pull/31968
• @andrew52522 made their first contribution in https://github.com/sgl-project/sglang/pull/30614
• @Lin-z-w made their first contribution in https://github.com/sgl-project/sglang/pull/30256
• @13524182838 made their first contribution in https://github.com/sgl-project/sglang/pull/31563
• @Hert4 made their first contribution in https://github.com/sgl-project/sglang/pull/32736
• @Oxygen56 made their first contribution in https://github.com/sgl-project/sglang/pull/32540
• @wirybeaver made their first contribution in https://github.com/sgl-project/sglang/pull/32115
• @JinuJeong made their first contribution in https://github.com/sgl-project/sglang/pull/31339
• @qiushixiaoyu made their first contribution in https://github.com/sgl-project/sglang/pull/29016
• @Broduker made their first contribution in https://github.com/sgl-project/sglang/pull/30240
• @vroomfondel made their first contribution in https://github.com/sgl-project/sglang/pull/31220
• @zkyue made their first contribution in https://github.com/sgl-project/sglang/pull/31128
• @ramm made their first contribution in https://github.com/sgl-project/sglang/pull/32896
• @ajtulloch made their first contribution in https://github.com/sgl-project/sglang/pull/33126
• @alumkal made their first contribution in https://github.com/sgl-project/sglang/pull/30971
• @jinzhenfan made their first contribution in https://github.com/sgl-project/sglang/pull/31221
• @Zhangmj0621 made their first contribution in https://github.com/sgl-project/sglang/pull/29173
• @ltaodream made their first contribution in https://github.com/sgl-project/sglang/pull/30177
• @guptaishaan made their first contribution in https://github.com/sgl-project/sglang/pull/32910
• @gitover22 made their first contribution in https://github.com/sgl-project/sglang/pull/32525
• @mrain made their first contribution in https://github.com/sgl-project/sglang/pull/33103
• @amd-oshkarav made their first contribution in https://github.com/sgl-project/sglang/pull/32046
• @CyberSecurityErial made their first contribution in https://github.com/sgl-project/sglang/pull/33065
• @zealoct made their first contribution in https://github.com/sgl-project/sglang/pull/33367
• @TobyMint made their first contribution in https://github.com/sgl-project/sglang/pull/33365

Full Changelog: https://github.com/sgl-project/sglang/compare/v0.5.16...v0.5.17

Highlights

574 PRs from 169 contributors.

DSpark: confidence-driven speculative decoding: A new speculative algorithm. It drafts semi-autoregressively in blocks, then sizes each verify window from the draft's own confidence instead of a fixed draft length. Reaches 383.7 tok/s at accept length ~5 on DeepSeek-V4-Pro, TP8 on B300 (bs=1). Enable with --speculative-algorithm DSPARK and SGLANG_RAGGED_VERIFY_MODE=compact; tune the block with --speculative-dspark-block-size (#30261, #31434, blog).

Inkling support: A 975B-parameter multimodal MoE with a 1M-token context. It mixes sliding-window, full and Mamba2 linear attention, and adds an NVFP4 MoE, optional vision/audio towers and native MTP. On Blackwell it reaches up to 71.7k tok/s input and 171.0 tok/s per-user decode. Verified on Blackwell TP4/TP8, H200 and AMD MI350X / MI355X (#31681, blog, cookbook).

Other new models added: LongCat 2.0 FP8, JetBrains Mellum v2, Pi0.5, plus diffusion support for LongLive 2.0.

UnifiedRadixTree is now the default for SWA, Mamba and DSA models. Replay SSM and Mamba int8 checkpoints are synced onto it, and a cache hit now resets only the state it used (#30468, #30636, #30626, #31643).

GLM-5.2 DSA cache layer split under prefill CP: KV and indexer cache layers are sharded across CP ranks. Each rank owns a disjoint layer range instead of all layers. That cuts per-rank KV memory by ~74% (0.77 to 0.20 GB/rank) at 8192 tokens on GLM-5.2-FP8, 78 layers, cp_size=4. Enable with --enable-dsa-cache-layer-split, which needs --enable-prefill-cp --cp-strategy interleave (#29421).

ReplaySSM Ring Spec-Verify (GDN): Drops the per-draft SSM snapshot. Speculative scratch goes from 11.5 GB to 1.8 GB per GPU (6.4x smaller) on Qwen3.5-35B-A3B at TP1, at accuracy and throughput parity. Opt in with --enable-gdn-replayssm-spec (default off; GDN with a linear draft chain only, --speculative-eagle-topk in {None, 1}), and tune the ring via --linear-replayssm-cache-len (#28695).

Linear attention on Blackwell (SM100): The first correct KDA MTP path. Its recurrent_kda decode kernel runs at 29.6 us vs 36.8 us for Triton (ncu, B=64). The full decode path reaches parity by B=128 and 1.35x at B=256, and is slower below that (#30113). Separately, GDN/KDA CuteDSL prefill fuses state I/O into the chunk-h kernel (#30169).

QServe and FBGEMM FP8 quantization are removed: the experimental QServe (QoQ) W4A8 and FBGEMM FP8 paths are gone. --fp4-gemm-backend cutlass goes too, along with the in-tree NVFP4 JIT kernels, so NVFP4 GEMM now requires FlashInfer (#31109, #30448).

Dependencies: flashinfer 0.6.14 (#29910), CuTe DSL 4.6.0 (#31714), sgl-kernel 0.4.5 (#31496), llguidance 1.7.6 (#31484).

Breaking Changes & Upgrade Notes

• The experimental QServe (QoQ) W4A8 and FBGEMM FP8 quantization paths are removed (per #28543): #31109
• CUTLASS FP8 blockwise deleted for SM90 / SM100, SM120 moved to JIT: #30438
• --fp4-gemm-backend cutlass is removed along with the in-tree NVFP4 JIT kernels, so NVFP4 GEMM now requires FlashInfer. Use auto, which picks flashinfer_cutedsl on SM100 and flashinfer_cutlass on SM120: #30448
• UnifiedRadixTree is now the default for SWA, Mamba and DSA models. A behavior change on those architectures: #30468
• Chunked input-logprob processing is now on by default to cap peak memory: #31498
• FA3 sparse mask kernels are off by default: #30356
• Legacy Sphinx docs/ removed; the Mintlify cutover is complete: #28964
• sglang.kernels namespace: kernels are relocated verbatim and only import paths change; public wrappers keep defaulting to the AOT sgl_kernel backend, so code reaching past them to internal paths must update (RFC #29630): #30044, #31582
• num_tokens_per_bs renamed to num_tokens_per_req across spec-decoding runners: #30977
• --enable-deepep-waterfill is renamed to --enable-waterfill with no deprecated alias, so existing launch commands fail with unrecognized arguments: #27350
• --optimistic-prefill-retries is renamed to --optimistic-prefill-attempts with no deprecated alias: #30951
• The SGLang-Diffusion post-training rollout endpoint now returns application/msgpack instead of JSON, with tensors as raw msgpack bytes rather than base64 (tensor_to_base64 / base64_to_tensor become tensor_to_bytes / bytes_to_tensor), so RL rollout consumers must be upgraded in lockstep with the server: #31565

Known Issues

• Temperature-0 nondeterminism under DP attention with breakable prefill CUDA graph. On the DSV4-Flash FP4 recipe, the idle-rank dummy extend introduced by #30898 perturbs real requests' logits, so identical temperature-0 requests can diverge. The guarding determinism test is disabled as a stopgap rather than fixed (#31125); not enabling breakable prefill CUDA graph avoids the path.
• A bump to flashinfer 0.6.15 was landed and reverted this cycle; this release pins 0.6.14 (#31502, #31625).
• Mamba track-boundary seqlen under the overlap scheduler was fixed and then reverted (#31369, #31622). The underlying issue is still open.
• CPU AMX optimizations for diffusion were reverted (#28527, #30716).
• GB300 CI jobs were temporarily disabled for runner availability during this cycle (#31764), so GB300 coverage rests on the cookbook's manual end-to-end validation.

Full release notes by category below.

New Model Support

Model	Type	PRs	Cookbook
Inkling	autoregressive	#31681	link
LongCat 2.0	autoregressive	#30275, #30320	link
JetBrains Mellum v2	autoregressive	#27375	wip
Pi0.5	vla	#30633	link
LongLive 2.0	diffusion	#27639	link

Landed this cycle but not yet usable end-to-end: MiniMax-M3 completes its four-part landing (#28715, begun in v0.5.14) but its cookbook still points at a dev image (#31819).

Inkling

• Add Inkling model support: #31681 ⭐
• Add Inkling cookbook: #31360
• [Docs] Inkling cookbook: mark B300/GB300 recipes verified, tune B300 MTP mem fractions: #31550
• [Cookbook] Inkling: add measured accuracy numbers to benchmark cards: #31823
• [Docs] Inkling cookbook: LoRA cells require --disable-prefill-cuda-graph: #31418
• Fix dropped Inkling reasoning at stream end: #31787
• [Spec] fix inkling multi layer mtp draft extend cuda graph: #32254 (cherry-picked as #32260)

GLM-5.2

• [Feature][GLM5.2] Add DSA Cache Layer Split under Prefill CP: #29421 ⭐
• support GLM-5.2 MTP index sharing with prefill CP: #30992
• [Fix] Stabilize GLM-5.2 MTP IndexShare across PD and CUDA graph replay: #30839
• [GLM5][MoE] perf: Write FlashInfer TRT-LLM MoE output directly: #28416
• Fix GLM/DeepSeek NVFP4 + flashinfer_trtllm long-context "!!!!" collapse (NaN routing): #31001
• [Docs] Update GLM5.2 Cookbook with LayerSplit usage: #31577

DeepSeek V4

• [DSA] Integrate Q8KV8 FP8 Sparse MLA Prefill into the DSA Backend (DeepSeek-V3.2): #30514
• [DeepSeek-V4] Enable non-paged indexer by default for large prefill chunks: #30140
• [Feature] Support DeepSeek-V4 Wint4Abf16 and Win4Afp8: #25763
• [DeepSeek-V4] Support BF16 Compress State for Online C128: #29609
• Implement SM120 DeepSeek V4 flashinfer_mxfp4 moe runner backend + TP2: #30272
• [DSV4] Remove per-step seqlen D2H from speculative to make overlap scheduler work: #30365
• [DSV4] Use BF16 instead of FP32 for indexer score computation: #30012
• [DSA] Fix top-k v2 emitting invalid indices under tie overflow / inf scores (IMA in FA3 sparse decode): #30645
• [DeepSeek-V4] Fix idle-rank dummy-extend sparse-prefill crash under DP breakable CUDA graph: #31705
• Fix nvfp4 online scale with pcg: #32246 (cherry-picked as #32259)
• Fix stale flashinfer-MLA fallback poisoning spec verify capture (trtllm_mla + tc_piecewise): #32288 (cherry-picked as #32346)

Speculative Decoding

• [Spec] Add DSpark: confidence-scheduled speculative decoding: #30261 ⭐
• [GDN] Support ReplaySSM Ring Spec-Verify: #28695 ⭐
• fa3/fa4: sync-free for all backends and phases: #29589
• fa3: sync-free eagle spec via fixed-window draft-extend metadata: #31364
• fa3: build the topk>1 verify replay page table on-device: #31381
• flashmla: sync-free spec via device-side draft-extend: #31090
• [Spec] DFlash: remove per-step host syncs so the CPU runs a full step ahead (spec-v2 overlap): #31468
• [Perf] Cache uniform ragged-verify layout for DSpark verify-all compact: #31434
• Support speculative decoding on CPU: #27862

Piecewise & Breakable CUDA Graph

• Enable breakable prefill CUDA graph for DP attention: #30898
• feat: enable piecewise prefill graph for Kimi K2.5/K2.7: #30889
• [Diffusion] Enable breakable CUDA graph (BCG) for diffusion DiTs: #27436

Attention Backends

• [KDA] Add FlashInfer SM100 KDA decode + MTP (target_verify) backend: #30113 ⭐
• [GDN/KDA] Fuse SM100 CuteDSL prefill state I/O into the chunk h kernel: #30169 ⭐
• [GDN] Auto-select FlashInfer GDN prefill on validated SM100 configs: #29734
• [Feature] Add FP4 KV Cache Design and support SM120 GPUs: #21601
• Fix KDA prefix caching under mamba extra_buffer and enable it for kimi_linear: #31474
• Fuse the preprocess kernels of trtllm-gen attention: #29690

MoE & Expert Parallelism

• [1/N] elastic-ep: Add runtime EP scale-up: #30164
• Support Waterfill with MegaMoE backend: #27350
• Support Flashinfer one-sided A2A + CuteDSL MoE for Nemotron Ultra: #28309
• Improve EPLB dispatch handling and diagnostics: #30646

Quantization

• Remove QServe and FBGEMM FP8 quantization: #31109 ⭐
• Delete CUTLASS FP8 blockwise for SM90 and SM100, move SM120 to JIT and add SwapAB: #30438
• Refactor FP4 quantization and remove deprecated JIT kernels: #30448
• [Quantization] add humming quantization kernel: #23754

Parallelism & Disaggregation

• [CP] Migrate MLA prefill CP (DeepSeek V3) to CP-v2 zigzag strategy: #31619
• Support MiMo V2.5 with zigzag context parallelism: #29972
• Support GPT-OSS zigzag CP with TRTLLM-MHA: #31732
• [DCP] Enable decode context parallel for Kimi K2.5 NVFP4: #31514
• [PDD] Add true request retraction for PDD: #25372
• [PD] Improve optimistic prefill: #30951
• [PD] Fix optimistic prefill inflight-queue hangs on parked/aborted reqs: #31075
• feat(grpc): support disaggregated generation requests: #30440
• [gRPC] Native server: launcher + HTTP + server args wiring (3/4): #23508
• feat: add native gRPC sidecar module launcher: #31076 (cherry-picked as #32074)

Scheduler & Runtime

• Using UnifiedRadixTree by default for SWA, Mamba, and DSA models: #30468 ⭐
• [Feature] Add --default-chat-template-kwargs server arg: #29579
• [Scheduler] Add SGLANG_MAX_NEW_TOKENS_LIMIT to cap per-request max_new_tokens: #22591
• Support priority request header override: #30811
• Align reasoning_effort schema across chat, tokenize, and responses: #31784
• Return top-p/top-k sampling mask/nucleus: #27408
• [Scheduler] Move the WAR barrier to right after each run_batch launch: #31687
• [Fix] Enable chunked input-logprob processing by default to cap peak memory: #31498
• [Refactor] Unify logprob results into a single LogprobResult and rename chunk env vars: #31733
• [dLLM] Make FDFO a framework capability for all dLLM algorithms: #27551

HiCache & Radix Cache

• [HiCache] Add the FlexKV storage connector: --enable-flexkv routes the KV cache through FlexKV's KVManager for host-tier offload, configured via --flexkv-config-file: #29701
• [HiCache] Add a client-side metadata cache for the HiCacheFile backend, bypassing directory traversal on lookups (SGLANG_HICACHE_FILE_BACKEND_ENABLE_METADATA_CACHE, off by default): #29716
• [HiCache] Optimize L2 mem allocation when cache miss in L3: #19320
• [HiCache] Optimize HiCache host pool free-list release: #30658
• [UnifiedTree] Sync Replay SSM: #30636
• [UnifiedTree] Sync mamba int8 checkpoint: #30626
• Reset only the used mamba state on unified radix cache: #31648
• Reset only the used mamba state on radix cache hit: #31643

LoRA

• [Diffusion] post_training: Add LoRA IPC weight sync via lora_merge mode: #31029
• Move LoRA cuda-graph buffers and logging into LoRAManager: #31151

Multimodal

• feat: unify multimodal feature transport: #30904
• vlm: batch cross-request vit encoding and reuse attention metadata: #24013
• [Multimodal] Support n>1 outputs for GLM-Image generation: #31027

Model Support & Optimizations

• Add DeepReinforce Ornith-1.0 to cookbook: #29404
• Fix MiMo-V2 on Blackwell: FA3 fallback and TP-aware audio weight loading: #31343
• Fix Ministral3 accuracy issue by aligning YaRN RoPE scaling with Transformers implementation: #31232
• Fix garbage output for bare-tekken Mistral checkpoints (e.g. Leanstral): #30396
• [Fix] Map reasoning_effort=low to Nemotron-3 Super low_effort + warn on unsupported levels: #30463

Kernel Library (sglang.kernels, RFC #29630)

• [Kernel] Introduce sglang.kernels namespace and migrate scattered triton_ops kernels (Phase 2): #30044
• [Kernel] Migrate scattered quantization, MoE, srt/layers, generic-attention, DSA/DSV4, linear-attention and vendored fla/mamba kernels (Phase 2.5, 1-7/7): #30784, #30786, #30787, #30789, #30792, #30793, #30795
• [Kernel] Decouple KernelBackend from device + device-based CapabilityRequirement: #31292
• [Kernel] Fill non-CUDA coverage: HIP (aiter/rocm-triton) + Ascend NPU backends: #31307
• [Kernel] Sweep decoupled scattered kernels into sglang.kernels.ops: #31582

SGLang-Diffusion

• [Diffusion] model: support fal Ideogram V4 Fast and Instant: #31177
• [Diffusion] SGLang backend for GLM Image AR. Step 1 - Separate server: #25381
• [Diffusion] Support SP for Krea-2: #29777
• [Diffusion] msgpack raw-bytes transport (drop base64/JSON): #31565

AMD / ROCm

• [AMD] Reuse fused FP8 KV cache write on standard aiter prefill/decode: #26852
• [AMD] Enable mamba-extra-buffer for Qwen3.5 on ROCm: #30359
• [AMD] [Fix] Fix --attention-backend triton work for DeepSeek MLA on MI355 (null-K + decode dispatch + RoPE): #30355
• [AMD] Fix DeepSeek MLA prefill shape mismatch on HIP eager fallback (missing mha_companion_layers): #31675
• [AMD] Remove ROCm page_first+kernel -> layer_first HiCache fallback: #30622
• [Fix] fix quickreduce acc error in cudagraph mode: #29508
• Fix ROCm fused KV and KDA paths: #31688
• cookbook(deepseek-v4): add MORI disagg backend for AMD + bump MI355X image: #30651

NPU / Ascend

• [NPU] Add support --pre-warm-nccl: #30312
• [NPU] use standalone group for moe ep: #29030
• [NPU] Add extra topk_weights input in deepep ll dispatch: #29480
• [NPU] Determine the topk norm_type through scoring_func: #31107
• [NPU] custom-ops adapt: #30731
• [MoE Refactor] [NPU] Refactor Ascend MoE implementation to reduce code duplication and align with community design: #25663
• [NPU][Quantization] Add W4A4 MXFP4 quantization support for Qwen3 Dense on Ascend NPU: #23795
• [Fix][NPU] Fix/Refactor routed scaling factor application in MoE routing: #31449
• [NPU] FIX CMB illusion of garbled characters acc problems, in prefix cache mtp scenarios: #31659

CPU / Intel / XPU

• [Intel GPU] DeepSeek V4 5/N, 9/N, 11/N, 12/N, 13/N: move fused indexer RoPE/Hadamard, paged MQA logits, silu_and_mul_clamp and V2 Compressor kernels onto sgl-kernel for XPU: #27873, #28046, #28059, #28428, #28439
• [Intel XPU] Enable (biased) grouped topk for xpu: #31126
• [XPU] Route topk_sigmoid and topk_softmax to AOT sgl-kernel-xpu symbols: #31038
• [CPU] add fused input proj for qwen3.5: #31171
• [CPU] improve silu performance by replacing fp32 div with rcp14: #31304
• Make UTs compatible for XPU: #27106
• [MLX] Honor --max-running-requests in the model runner stub: #30547

Dependencies

• [Dep] Upgrade flashinfer to 0.6.14: #29910 ⭐
• Bump CuTe DSL to 4.6.0: #31714 ⭐
• chore: bump sgl-kernel version to 0.4.5: #31496, #31618
• Upgrade llguidance to 1.7.6: #31484

Full Changelog: v0.5.15...v0.5.16

New Contributors

• @linhu-nv made their first contribution in https://github.com/sgl-project/sglang/pull/29701
• @averyjones4 made their first contribution in https://github.com/sgl-project/sglang/pull/29404
• @tyuchn made their first contribution in https://github.com/sgl-project/sglang/pull/29716
• @hdt98 made their first contribution in https://github.com/sgl-project/sglang/pull/29275
• @connorcarpenter15 made their first contribution in https://github.com/sgl-project/sglang/pull/30440
• @wangjiaxin99 made their first contribution in https://github.com/sgl-project/sglang/pull/30265
• @htzo made their first contribution in https://github.com/sgl-project/sglang/pull/27862
• @ICENacl made their first contribution in https://github.com/sgl-project/sglang/pull/28982
• @spandantiwari made their first contribution in https://github.com/sgl-project/sglang/pull/25467
• @rwang5203 made their first contribution in https://github.com/sgl-project/sglang/pull/27576
• @Junjie650 made their first contribution in https://github.com/sgl-project/sglang/pull/30408
• @Hayden727 made their first contribution in https://github.com/sgl-project/sglang/pull/27551
• @starkwj made their first contribution in https://github.com/sgl-project/sglang/pull/30747
• @ZYHowell made their first contribution in https://github.com/sgl-project/sglang/pull/30828
• @yz-wqf made their first contribution in https://github.com/sgl-project/sglang/pull/30846
• @auroter made their first contribution in https://github.com/sgl-project/sglang/pull/30331
• @N3u0ns made their first contribution in https://github.com/sgl-project/sglang/pull/28113
• @jinzhen-lin made their first contribution in https://github.com/sgl-project/sglang/pull/23754
• @shadeMe made their first contribution in https://github.com/sgl-project/sglang/pull/27375
• @ankith117 made their first contribution in https://github.com/sgl-project/sglang/pull/31143
• @hunhokim made their first contribution in https://github.com/sgl-project/sglang/pull/30351
• @AuFlow made their first contribution in https://github.com/sgl-project/sglang/pull/30621
• @zhihengy made their first contribution in https://github.com/sgl-project/sglang/pull/30036
• @jorgeantonio21 made their first contribution in https://github.com/sgl-project/sglang/pull/30182
• @sunjiweiswift made their first contribution in https://github.com/sgl-project/sglang/pull/31140
• @guzekai01 made their first contribution in https://github.com/sgl-project/sglang/pull/31185
• @IzacharyI made their first contribution in https://github.com/sgl-project/sglang/pull/26852
• @jojoakm made their first contribution in https://github.com/sgl-project/sglang/pull/30535
• @beef9999 made their first contribution in https://github.com/sgl-project/sglang/pull/19320
• @nzr-niu made their first contribution in https://github.com/sgl-project/sglang/pull/31174
• @Safiullah136 made their first contribution in https://github.com/sgl-project/sglang/pull/31584
• @twb1235 made their first contribution in https://github.com/sgl-project/sglang/pull/25213

v0.5.15.post1 includes a few patches, mostly for GLM 5.2

• #30454 #30627: Fix DSA model launching on non Cuda/HIP devices
• #30858: Fix flashinfer dependency on Cuda 12 images
• #31001: Fix NaN outputs caused by flashinfer trtllm FP4 MoE kernels on long input
• #30839: Fix GLM 5.2 IndexShare on PD disaggregation setting
• #30992: Fix GLM 5.2 IndexShare on Context Parallel setting

Highlights

GLM-5.2 NVFP4, tuned for production: We took time this cycle to tune GLM-5.2 NVFP4 on Blackwell for optimized production serving. It now runs at 500+ tok/s/user on 8x B300, 450 on 4x GB300 (bs=1). Run GLM-5.2 with our cookbook.

• Spec V2 by default: zero-overhead scheduling via CUDA-graphable DSA draft-extend, dropped D2H/H2D syncs, fused metadata ops. +11% end-to-end TPS (#29413, #29343, #29166, #29077).
• IndexShare MTP: reuses the indexer top-k across draft steps, up to 1.9x lower draft-step cost at long context (#29959, #29787, #29654).
• TopK V2: fuses top-k selection with the page-table transform, runtime k up to 2048 (#30274, #26788).
• Indexer prologue fusion: 12 kernels to 4, ~8% faster decode at bs=1 (#27705).
• GEMM: shape-specialized JIT router GEMM + CuteDSL BF16 GEMM for Blackwell (#21531, #30117).
• FlashInfer autotune: now covers draft-model graphs (#29595).

New Model Support: Hunyuan 3 (Hy3), Hierarchical Reasoning Model (HRM-Text), NVIDIA LocateAnything-3B, Baidu Unlimited-OCR, JoyEcho multi-shot A/V, plus Qwen3.6 NVFP4 support.

Native web search (Exa): Built-in web_search support backed by Exa (#29342).

Breakable CUDA Graph on by default: Breakable CUDA Graph is now the default capture path, reducing per-step kernel-launch overhead (#29458); full CUDA Graph support for the prefill phase lands as experimental (#27988).

Linear-attention kernels (KDA / GDN): New FlashKDA prefill backend for safe-gate KDA linear attention (#29472), plus ReplaySSM buffered output-only decode for linear attention (#28451).

FlashInfer A2A for routed MoE: Adds FlashInfer all-to-all with the flashinfer_trtllm_routed MoE runner (#22394).

DeepSeek-V4 Optimization:

• Optimizes C128 state-pool allocation using the request state pool (#28612).
• FlashMLA sparse prefill is now enabled by default for DeepSeek-V4, reaching >10% throghput gain on long context. (#29775).
• Non paged indexer support for long context prefill, with >5% e2e throughput gain.(#29619).

Decode Context Parallelism: decode context parallelism lands for MLA models, including DeepSeek V3 and Kimi K2 series (#14194).

Dependency upgrades: transformers bumped to 5.12.1 (#29393); tvm-ffi / sgl-deep-gemm / tilelang upgraded (#29554). and . See the DeepSeek-V4 cookbook.

Full release notes by category below.

New Model Support

• Hunyuan 3 (Hy3): #30201 (cookbook)
• Hierarchical Reasoning Model (HRM-Text): #27887
• NVIDIA LocateAnything-3B: #28958
• Baidu Unlimited-OCR: #29186 (cookbook)
• JoyEcho multi-shot A/V: #27420 (cookbook)
• Qwen3.6 ModelOpt mixed NVFP4: #27906, #29905 (cookbook) ⭐

GLM-5.2

• [BCG][GLM5] perf: BCG support and prefill enhancements: #27053 ⭐
• [CI] Add GLM52 NVFP4 MTP B200 tests: #30021
• [cookbook] GLM-5.2 NVFP4 B300: TP8 recipe + 3 strategies: #29557
• [Cookbook] GLM-5.2: tune GB300 NVFP4 recipes + fill benchmarks: #29486
• [cookbook] drop redundant serve flags (GLM-5.2) + fix M3 page-size note: #28731
• [Docs] Add NVFP4 quantization to GLM-5.2 cookbook: #29380
• [DSA][GLM5.2] Index Share for MHA: #29959
• [GLM-5] Tune the threshold of router GEMM: #29470
• [Spec] Anchor GLM-5.2 MTP IndexShare topk on the draft-extend step: #29787
• Bypass legacy GLM DSA layer types validation: #29454
• docs: add B200 NVFP4 recipes + benchmarks to GLM-5.2 cookbook: #29674
• docs: add PD disaggregation to GLM-5.2 cookbook playground: #29544
• docs(cookbook): add AMD MI300X/MI325X/MI355X support for GLM-5.2: #28471
• Fuse the DSA (V3.2, GLM-5.x) indexer Q/K paths into single kernels: #27705 ⭐
• glm5.2 on ascend doc (new version): #29828
• Support JIT fused A GEMM (MLA down projection) and support GLM-5 hidden size, SM120: #27397
• Update GLM-5.2 B300 and GB300 NVFP4 cookbook settings: #29466
• Update GLM tests to 5.2 and delete redundant tests: #29686
• [AMD] [GLM5] Guard cuda_runtime.h for ROCm in fused_metadata_copy: #29373
• [AMD] [GLM5] Mark EAGLE verified on MI300X/MI325X (gfx942) in GLM-5.1 cookbook: #29313
• [AMD] [GLM5] GLM-5.1 MXFP4 (MI355X) + enable EAGLE for gfx950 in cookbook: #29194
• [AMD] [GLM5] Add opt-in Triton fp8 sparse-MLA prefill kernel for gfx950: #28975
• [AMD] [GLM5] skip redundant -inf pre-fill of HIP indexer MQA-logits: #28757

DeepSeek V4

• [DeepSeek V4] Enable FlashMLA sparse prefill by default: #29775 ⭐
• [DeepSeek-V4] Add an opt-in non-paged indexer for long-context prefill: #29619 ⭐
• [DSA] Fold page-table into fused top-k v2 (decode): drop page_size=1 expansion: #30274
• [JIT Kernel] DeepSeek-V4 DSA indexer: faster top-k + page-table transform (runtime k <= 2048): #26788 ⭐
• [Cherry-pick to release/v0.5.15] [DSA] Fix IMA in fused top-k v2: write all output slots on tie overflow (#30512): #30559
• [Cherry-pick to release/v0.5.15] [DSV4] perf: Make FP8 quant output tensor contiguous (#27926): #30449
• [Cherry-pick to release/v0.5.15] [DeepSeek-V4] Enable non-paged indexer by default for large prefill chunks (#30140): #30436
• [Cherry-pick to release/v0.5.15] [DSA] Re-enable fused top-k v2 for MTP: clamp padded-row seq_lens to >= 0 (#30378): #30427
• [AMD] Improve performance of DSV4 in high concurrency: #28938
• [AMD] DSV4 aiter reduce-scatter decode: #29103
• [AMD][DSV4] Remove per-batch D2H syncs in MTP to avoid bubbles between 2 batches: #29420
• [AMD][DeepSeek V4] Fix default FlashMLA sparse prefill off on ROCm/HIP: #29982
• [AMD] Fix DeepSeek V4 MTP accuracy issue: #30333
• [AMD] Fix dsv4 indexer dtype dispatch on gfx950: #29479
• [AMD] Cap DSV4 Flash max_total_num_tokens: #30313
• [AMD] Fix DeepSeekV4 server cutlass error: #30374
• [Intel GPU] DeepSeek V4 10/N : Add sqrtsoftplus support to fused_topk_torch_native: #28048
• [Intel GPU] DeepSeek V4 7/N: Support fused_rope_inplace on XPU using triton: #27915
• [Intel GPU] DeepSeek V4 3/N: Support hc_split_sinkhorn on XPU using sgl_kernel: #27783

Speculative Decoding

• [MoE] Fix moe_fused_gate out-of-range expert id on all-NaN rows (fixes eagle_dp_attention crash): #30079
• [perf] tiny optimize select_index op for draft extend: #29078
• [perf] simplify _apply_cuda_graph_metadata for draft extend in trtllm_mla backend: #29077
• [Spec] Enable FlashInfer autotune for spec draft: #29595
• Speculative decoding support on XPU: #23180
• (perf): Shard Kimi-K2.5 Eagle3 draft fc + symm-mem AG: #29223
• [XPU] Unbreak stage-b: re-add --disable-decode-cuda-graph, quarantine EAGLE3 parity: #30048

Piecewise & Breakable CUDA Graph

• [Experimental] Full Cuda Graph Support for Prefill: #27988 ⭐
• Disable dsr1 prefill cudagraphs by default: #28053
• Enable Breakable Cuda Graph as Default: #29458 ⭐
• [AMD] Enable BCG on ROCm + route aiter prefill via MHA during PCG/BCG capture for Kimi-2.5: #27833
• [XPU] Enable XPU graph support (decode full-graph + prefill tc_piecewise): #29053

Attention Backends

• [GDN][KDA] ReplaySSM buffered output-only decode for Linear Attention: #28451 ⭐
• [KDA] Add FlashKDA prefill backend for safe-gate KDA linear attention: #29472 ⭐
• [KDA-Pilot] Add LTX2 QKNorm split-RoPE CUDA fast path: #29708
• [KDA-Pilot] Add diffusion residual-gate CUDA fast path for LTX2: #29361
• [KDA-Pilot] Add diffusion causal Conv3D cat-pad CUDA fast path for Cosmos3: #29281
• [MoE] Consolidate ungrouped + grouped gate/topk onto one Triton router (#26771) — faster than AOT on B200/H100/H200, at parity with flashinfer: #29771
• [NVIDIA] Support flashinfer a2a with flashinfer_trtllm_routed moe: #22394 ⭐
• Re-enable SM90 FlashInfer allreduce fusion with safe backend defaults: #28789
• [CPU] optimize GDN prefill performance: #29117
• [NPU] perf: precompute mamba conv-state track indices once per batch: #29105

MoE & Expert Parallelism

• [JIT Kernel] Triton moe fused gate: #25835
• [MoE] Retire the AOT moe_fused_gate / kimi_k2_moe_fused_gate gate kernels (#26771): #29997
• [Perf][Kernel] Fuse SiLU+Mul into NVFP4 Expert Quantization for CUTLASS MoE: #18612
• Add GB10 FP8 fused MoE Triton config: #25665
• [AMD] Fuse shared-expert append + DeepEP remap into one Triton kernel: #28450
• [AMD] fix(moe): correct fused shared-expert scaling on aiter/DeepEP path (mori all-to-all): #28237
• [AMD] Implement QuarkW4A8MXFp4MoE to support amd/gpt-oss-120b-w-mxfp4-a-fp8: #27204
• [AMD][MORI-EP] Skip LocalExpertCount kernel in decode graph when not recording: #30302
• [NPU] [DOC] add missing DEEP_NORMAL_MODE_USE_INT8_QUANT for w8a8+deepep scenarios: #29937

Quantization

• [diffusion] Add Qwen-Image ModelOpt NVFP4 support: #28928
• [Kernel] Add SM90 Q8KV8 FP8 Sparse MLA Prefill JIT Kernel with Tests and Benchmark: #25751
• [weight checker] refactor: add precision branch; allow ULP quant err; used chunked compare: #28974
• Add Intel Quantization Support in SGLang: #18139
• Fix gfx95 bpreshuffle FP8 activation scale layout: #29275
• :sparkles: [llm][npu][quant] Add W4A8 MXFP quantization support for Qwen3 Dense on Ascend NPU: #23650
• [AMD] Gate broken CK block-FP8 GEMM shapes to aiter-triton-GEMM to fix ROCm 7.0 Qwen3.5 accuracy: #29918

Parallelism & Disaggregation

• [feature] implement dcp for deepseek_v2: #14194 ⭐
• [HiCache] Add NIXL FILE cache cleaner: #28258
• hisparse: support NIXL DRAM KV destinations for HiSparse: #27563
• [AMD]: Enable NIXL PD disaggregation for ROCm(1/n): #28348
• [AMD]: docker(rocm) bump Mooncake to latest main + enable multi-protocol: #27730
• [AMD] Support triton backend decode context parallel for Qwen3.5: #25090
• [AMD] Fix AITER custom all-gather CUDA-graph capture crash under torch_memory_saver: #30557

Scheduler & Runtime

• [Perf] Overlap result D2H copy with the next forward step: #29075
• [refactor] Add a read-through server_args accessor to RuntimeContext (stack 1/15): #30063
• [scheduler] Add scheduler metrics reporter init hook: #29535
• Add native Exa-backed web_search support: #29342 ⭐
• Add scheduler metrics extension hooks: #29207
• feat(metrics): add Prometheus metrics for the EPD encoder server: #27564
• Support DP-aware PD router dispatch: #26245

HiCache & Radix Cache

• [HiCache] Optimize HiCache hash generation with bulk token byte conversion: #28287
• [HiCache][AMD] Add UMBP tiered DRAM + SSD L3 storage backend with hugepage host allocator: #25377
• [optimize] fix swa eviction boundary for unfinished inserts: #29350
• Optimize C128 state pool allocation using request state pool: #28612 ⭐

Multimodal

• [EPD] Optimize multimodal global cache with paged embedding pool: #28441
• [VLM] Qwen3-VL / Moss-VL ViT preprocessing optimizations: #28940 ⭐
• Add MiMo V2.5 Blackwell vision FA4 recipe: #29253

Model Support & Optimizations

• [NVIDIA] Support TF32 matmul to improve MiniMax gate gemm performance: #22744
• [trtllm_mha] Fuse cuda-graph metadata rebuild into one triton kernel: #29843
• Add fused EH norm for DeepSeek NextN: #29667
• Add Laguna XS.2.1 DFlash support to SGLang: #29446
• Fused QK GemmaRMSNorm + RoPE + gate kernel for Qwen3.5: #28320
• perf(triton): avoid per-step D2H .item() sync in cuda-graph loc translate: #29921
• Support Cutedsl BF16 GEMM JIT kernel: #30117

SGLang-Diffusion

• [diffusion] perf: add unified SP shard helpers and zero-copy tail-pad attention: #30107
• [diffusion] perf: tp-shard every text/image encoder across the full DiT replica (any parallelism): #30086
• [diffusion] feat: add LingBot realtime prompt, KV window, and lazy VAE controls: #30040
• [diffusion] feat: performance_mode=speed enables torch.compile by default: #30016
• [diffusion] feat: add --offload-during-compile to fit max-autotune on tight-memory GPUs: #29862
• [diffusion][cache-dit] support Krea-2 + run-driven has_separate_cfg: #29688
• [diffusion] feat: support cache-dit for Ideogram 4: #29631
• [Diffusion] Add Krea 2 support: #29052
• [diffusion] fix: add profiling support and fix VBench dataset handling in bench_offline_throughput: #27704
• [diffusion] rl: add sleep/wake support for diffusion engine: #22659
• [AMD][diffusion] Disable layernorm torch.compile decorator in eager mode on ROCm to avoid memory-access fault: #29673

AMD / ROCm

• [mori] Add a combine-kwargs hook and use_external_inp_buf plumbing: #29097
• [AMD] Fuse topk padded-token masking into a single Triton kernel: #28084

NPU / Ascend

• [NPU] [DOC] Update arguments detail to NPU support features page: #30328
• [NPU] [DOC] Update deterministic inference feature support status to A2, A3: #29632
• [NPU]GLM-4.7-Flash optimize with fused kernels: #29509
• [NPU] [DOC] Add environment prerequisites to model tutorials: #29293
• [NPU] Support fsdp for rl_on_policy_target: #29128
• [NPU] adapt_fused_rope_qk_mqa_optimize: #28872

CPU / Intel / XPU

• [qwen3.5][XPU]Add XPU support for set_embed_and_head and fused QK RMSNorm kernel: #27870
• [Apple Silicon] Add labeler config: #29908
• [CPU] add fused_qk_gemma_norm and refactor norm kernel implementation: #30216
• [CPU] enable fused_sigmoid_mul on CPU device: #29378
• [Intel GPU] add pytorch profiling support for XPU in bench offline throughput and enhance num steps: #28308
• [XPU] Remove redundant xpu graph backend and make xpu graph opt-in by default: #29911

Dependencies

• [Deps] Bump transformers to 5.12.1: #29393 ⭐
• feat(sgl-kernel): add InfLLM v2 attention kernels: #29383
• Upgrading tvm-ffi/sgl-deep-gemm/tilelang: #29554 ⭐

All PRs included in this release: v0.5.14...v0.5.15

New Contributors

Approximate list from git history; GitHub's "Generate release notes" produces the canonical, deduped list.

• Augusto Yao made their first contribution in #14194
• Wang, Mengni made their first contribution in #18139
• meinie made their first contribution in #21531
• ANSHUMAN TRIPATHY made their first contribution in #23180
• Rohit Harkhani made their first contribution in #25071
• TzZtzt made their first contribution in #25153
• Daniel Stokes made their first contribution in #26255
• Jiajun Li made their first contribution in #26980
• Kaixi made their first contribution in #27053
• Stanley Winata made their first contribution in #27204
• hhhh1252023 made their first contribution in #27433
• Oxana Korzh made their first contribution in #27835
• yifei wu made their first contribution in #27887
• chengcuiping made their first contribution in #28401
• Raiden Makoto made their first contribution in #28455
• Martin Hua made their first contribution in #28481
• Tai An made their first contribution in #28503
• a60124901 made their first contribution in #28586
• Zhihao Wang made their first contribution in #28676
• toufupi made their first contribution in #28770
• Yuankai Chen made their first contribution in #28787
• Anusha Pant made their first contribution in #28952
• Jyothirmai Kottu made their first contribution in #28958
• qyb233 made their first contribution in #28980
• Alex Tumanov made their first contribution in #28996
• hirakunaramuka2 made their first contribution in #29004
• IvanShan177 made their first contribution in #29102
• Shijin Zhang made their first contribution in #29161
• Aditya Kamat made their first contribution in #29186
• jonah-berman made their first contribution in #29342
• Feng Yao made their first contribution in #29350
• Feng Yao made their first contribution in #29352
• cauphe made their first contribution in #29383
• Siming Deng made their first contribution in #29440
• adamkbaranowski made their first contribution in #29446
• bef0rewind made their first contribution in #29570
• SSSunzt made their first contribution in #29571
• Jzz1943 made their first contribution in #29631
• sushil Dubey made their first contribution in #29672
• YukioZzz made their first contribution in #29756
• san-tian made their first contribution in #29798
• Pranjal Shankhdhar made their first contribution in #29843
• NOOB made their first contribution in #30181

Full Changelog: v0.5.14...v0.5.15

Highlights

New Model Support: GLM-5.2, LiquidAI LFM2.5, Kimi-K2.7-Code, Poolside Laguna-M.1, DiffusionGemma, Zyphra ZAYA1, MiMo-V2-ASR

DeepSeek-V4 on GB300 since Day 0: 5x higher throughput at the same interactivity, serving DeepSeek-V4 on NVIDIA GB300 with SGLang (blog).

Waterfill & LPLB MoE load balancing: Two dispatch-time load-balancing methods for DeepEP expert parallelism: Waterfill for shared-expert dispatch and LPLB for redundant expert replicas, improving throughput for DeepSeek-V3/R1 and DeepSeek-V4 (blog).

KDA CuteDSL prefill kernel on Blackwell (SM100): New CuteDSL prefill kernel for Kimi-Linear (KDA), 1.08-1.52x faster than the Triton path via a reusable scratch workspace, plus a cuda-graph padding fix (#27488); see the Kimi-Linear cookbook.

Linear-attention prefix-cache memory savings: An int8 checkpoint pool stores recurrent states compactly in the Mamba radix cache, substantially increasing prefix-cache capacity for KDA / GDN models (#28185); the speculative conv-window intermediate cache is deduplicated with a sliding-window layout, halving its footprint with no numerical change (#28302).

LPLB: linear-programming load balancer for MoE expert parallelism: Balances token routing across redundant expert replicas by solving a per-layer LP; opt-in via --ep-dispatch-algorithm=lp, default behavior unchanged (#24515).

MSCCL++ integration & MNNVL allreduce fusion: MSCCL++ migrates to the upstream mscclpp Python package (Executor + DSL compiler) with auto-tuned collectives for TP=8 single-node and TP=16 two-node (#22734); FlashInfer fused allreduce + residual + RMSNorm re-enables an MNNVL backend behind --flashinfer-allreduce-fusion-backend (auto / trtllm / mnnvl), fixing the piecewise-CUDA-graph interaction (#23402).

Nemotron DP attention + MTP: Data-parallel attention for the hybrid Nemotron-H (Mamba2 + full attention + MoE), plus MTP support (#24955); see the Nemotron 3 Ultra cookbook.

AMD: breakable CUDA graph on ROCm/HIP: The breakable CUDA graph execution path now runs on AMD GPUs (#28173).

NVFP4 MoE for DeepSeek-V4: Adds an NVFP4 MoE quantization path for DeepSeek-V4 on Blackwell for higher MoE throughput; enable with --moe-runner-backend flashinfer_trtllm_routed (#25820); see the DeepSeek-V4 cookbook.

DeepSeek-V4 decode & quantization optimizations: FP8 group quantization now emits power-of-two (UE8M0) scales directly from the per-token group-quant kernel, dropping a separate rounding pass (#26766); MLA decode q-heads are padded to 64 under attention-TP so FlashMLA dispatches the ~2x cheaper head64 kernel instead of head128 (#27954); the MHC prenorm kernel is prewarmed at startup to remove the first-run JIT slowdown on a fresh server (#27986); and BF16 mixed-dtype compression states are supported on the C4 / C128 paths (#27277); see the DeepSeek-V4 cookbook.

Full release notes by category below.

New Model Support

• GLM-5.2: #28437 (cookbook)
• LiquidAI LFM2.5: #27409 (cookbook)
• Kimi-K2.7-Code: #28064 (cookbook)
• Poolside Laguna-M.1: #28661, #28400 (cookbook)
• DiffusionGemma: #27824 (cookbook)
• Zyphra ZAYA1: #26347 (cookbook wip)
• MiMo-V2-ASR: #26278 (cookbook wip)

DeepSeek V4

• [NVIDIA] Support NVFP4 MoE for DeepSeek-V4: #25820
• [DeepSeek-V4] Fuse UE8M0 scale rounding into FP8 group quantization: #26766
• [NPU] Add Ascend NPU support for DeepSeek-V4: #25144
• Deepseek v4: support mixed dtype compression states: #27277
• [AMD] Feat: Add prefill context parallel support for deepseek v4 unified kv attention: #27928
• DeepSeek-V4 Online Compress support MTP: #26471
• [dsv4] Pad MLA decode q-heads to 64 (not full n_heads) for FlashMLA head64 kernel: #27954
• [dsv4] Prewarm MHC prenorm kernel at startup: #27986
• [LoRA] Support DSA indexer LoRA targets for GLM-5.1 / DeepSeek-V3.2-family models: #28110
• Add DeepSeek V4 MTP acceptance length checks: #28098

Speculative Decoding

• [Spec] Add sync-free fast_prefill_plan for EAGLE draft-extend CUDA graph: #28854
• [Spec] Support FlashInfer CUDA graph for EAGLE draft-extend: #28782
• [mtp] add rejection sampling for speculative decoding: #26312
• [NPU] Add MTP support for GLM-4.7-Flash: #28516
• Dflash add sliding window attention draft layer support: #27469
• Support Nemotron DP attention and MTP: #24955
• [Feature] [Ngram spec] Support ngram spec v2: #17260

Piecewise & Breakable CUDA Graph

• [AMD] Make breakable CUDA graph run on ROCm/HIP: #28173
• Dflash piecewise cuda graphs support: #27468

Attention Backends

• [Cookbook] Nemotron3-Ultra: Add mamba-backend and SSM dtype flags: #28675
• [Mamba][GDN] Deduplicate spec conv-window intermediate cache via sliding window layout: #28302
• [GDN][KDA][mem_cache] int8 checkpoint pool for the linear-attn prefix cache: #28185
• [diffusion] feat: use LocalAttention for mistral3 encoder: #28176
• [NPU] Add Gemma4 Sliding Window Attention support on Ascend backend: #26147
• [AMD] Fuse sigmoid + mul attention output gate into single Triton kernel: #27630
• [AMD] Enable fused GDN QKV split Triton kernel on HIP: #27583
• [KDA] Add CuteDSL Prefill Kernel on SM100: #27488
• [AMD] Add unified kv attention support in dpsk-v4: #27380

MoE & Expert Parallelism

• Add GB10 FP8 fused MoE Triton config: #25665
• Support asymmetric compressed-tensors MoE: #27690
• LPLB: linear-programming load balancer for MoE expert parallelism: #24515
• [AMD] Fuse sigmoid + mul into single Triton kernel for shared expert gating: #27636
• [quantization] NVFP4 MoE: split fused w13 gate/up global scales: #27588
• [Apple Silicon] [MLX] Fuse SwiGLU activation into gate gather_qmv for SwitchGLU MoE blocks: #26188
• [DeepSeek V3] Defer moe finalize and fused it with main stream add: #27720

Quantization

• :sparkles: [llm][npu][quant] Add W8A8 MXFP8 quantization support for Qwen3 Dense on Ascend NPU: #22352
• Implement online nvfp4 quantization: #26083

Parallelism & Disaggregation

• Add Mooncake group semantics: #26574
• [cookbook] Laguna-M.1: add PD disaggregation section: #28737
• [2/n] [CP] Add context parallel strategy abstractions: #27313
• [AMD] Support unified_kv_triton for disaggregation: #27935
• Add bucketed multi-dir layout for NIXL file storage: #27672
• Add EPD disaggregated encode tracing: #25994

Scheduler & Runtime

• [core] Gate the overlap WAR barrier on forward reads to recover decode throughput: #28363
• [Feature] Add graceful scheduler shutdown; free hisparse host buffer on exit: #28779
• Support MPServer and embedded server for granian to enable muti tokenizer worker: #28573
• Add get_parallel(): a structured accessor for parallel-topology state: #28567
• Support GLM-4.7 function calling via structural tags: #28149
• Add SGLANG_ENABLE_WAR_BARRIER to force-enable the overlap scheduler WAR barrier on non-CUDA (e.g. AMD): #27967
• [router] Add request/TTFT/worker metrics + Grafana dashboard to experimental sgl-router: #27591

HiCache & Radix Cache

• [HiCache]Support hybrid pool staged H2D kernel: #28434
• [HiCache & Bench] add cache hit breakdown in bench_serving: #22053
• [HiCache]Asymmetric pool support direct backend: #28446
• Support HiCache for MiMo-V2 models (1/N): #27378
• [UnifiedTree]: HybridModel launches HiCache via UnifiedTree by default.: #27759
• [HiCache] Add opt-in LRU eviction to file storage backend (CP-aware): #26670

LoRA

• [diffusion] perf: merge LTX-2 stage-1 distilled LoRA into the base in original mode: #28594

Multimodal

• Report multimodal (image/audio/video) token counts in usage.prompt_tokens_details: #27122
• Eliminate CUDA syncs in VLM embed path: #26082

Model Support & Optimizations

• Add dflash gemma4 support: #27471
• [NPU] Add MiMo-V2-Flash manual testcases: #28223
• Add mimo best practice: #27665
• [AMD][Perf] Fuse QK RMSNorm + gate extraction Triton kernel for Qwen3.5 on HIP: #27656
• Mistral3 add tensor parallel support for diffusion text encoder: #25950

SGLang-Diffusion

• Shard hunyuan text tokens under sp: #28319
• Shard text when using sp in flux.1/2: #27066
• Use srt custom allreduce for tp groups: #28324
• Optimize causal conv3d vae padding: #28204
• Persist torch.compile inductor/triton cache across restarts: #28205
• FLUX: fuse FeedForward GELU into up-proj GEMM (cublasLt epilogue): #28166
• Use regional torch.compile (compile_repeated_blocks) for DiT of diffusers backend: #28193
• Add --warmup-mode enum server arg: #28184
• Enable spatial-shard vae decode across GPUs: #28071
• Enable vae parallel decode with cfg-parallel: #27875
• Optimize flux1 tensor parallel sharding: #27826
• Progressive resolution growing for Ideogram 4 via GPU DCT upsampling with up to 1.56× speedup: #27736
• Use fused w8a8 kernel for Ideogram4 weight-only linear as an opt-in: #27590
• Run LTX-2 VAE decode in channels_last_3d (faster decode, lower peak memory): #27431
• Rl: extract post-training weight apis into mixins and add tensor update/checker paths: #22817

AMD / ROCm

• [AMD][DFlash] Enable Fused KV Materialization: #27854

NPU / Ascend

• [NPU] Add head_dim=256 to _can_use_tnd whitelist: #28635
• [NPU] Add NPU fallback for fused Triton gating kernels: #28293
• [NPU] [DOC] Update server arguments to NPU support features page: #28083

CPU / Intel / XPU

• [MLX] Add Metal profiling hooks to server profiler: #28122
• [Intel GPU]Add sycl mrope pass for xpu device: #27646

Dependencies

• sgl-kernel 0.4.3 → 0.4.4: #28556
• tokenspeed_mla 0.1.1 → 0.1.7: #28116, #28759
• Ray minimum version → 2.55.1: #27724
• pytorch-xpu → 2.12: #27133

Security

No security-tagged PRs in this release.

All PRs included in this release: https://github.com/sgl-project/sglang/compare/v0.5.13...v0.5.14

New Contributors

• @caiomcbr made their first contribution in https://github.com/sgl-project/sglang/pull/22734
• @LJL36 made their first contribution in https://github.com/sgl-project/sglang/pull/27550
• @xythink made their first contribution in https://github.com/sgl-project/sglang/pull/23802
• @gq112 made their first contribution in https://github.com/sgl-project/sglang/pull/25980
• @weizhoublue made their first contribution in https://github.com/sgl-project/sglang/pull/24401
• @BiggieW made their first contribution in https://github.com/sgl-project/sglang/pull/26320
• @yuchengliu1 made their first contribution in https://github.com/sgl-project/sglang/pull/27133
• @nbarzilie made their first contribution in https://github.com/sgl-project/sglang/pull/26908
• @ChengYao-amd made their first contribution in https://github.com/sgl-project/sglang/pull/26347
• @HZY-Wade made their first contribution in https://github.com/sgl-project/sglang/pull/26670
• @sigama-w made their first contribution in https://github.com/sgl-project/sglang/pull/27665
• @vinayK34 made their first contribution in https://github.com/sgl-project/sglang/pull/27157
• @luoroger37 made their first contribution in https://github.com/sgl-project/sglang/pull/27779
• @zalcit made their first contribution in https://github.com/sgl-project/sglang/pull/26278
• @xbfs made their first contribution in https://github.com/sgl-project/sglang/pull/26351
• @oulgen made their first contribution in https://github.com/sgl-project/sglang/pull/27967
• @Joectwm made their first contribution in https://github.com/sgl-project/sglang/pull/25994
• @liuxpro made their first contribution in https://github.com/sgl-project/sglang/pull/27866
• @Zhichenzzz made their first contribution in https://github.com/sgl-project/sglang/pull/24955
• @zqzten made their first contribution in https://github.com/sgl-project/sglang/pull/28013
• @Sunt-ing made their first contribution in https://github.com/sgl-project/sglang/pull/28088
• @Ibrahim2595 made their first contribution in https://github.com/sgl-project/sglang/pull/28078
• @evanderfff123-boop made their first contribution in https://github.com/sgl-project/sglang/pull/27913
• @JaredforReal made their first contribution in https://github.com/sgl-project/sglang/pull/26902
• @kingjameschan made their first contribution in https://github.com/sgl-project/sglang/pull/25975
• @HumphreySun98 made their first contribution in https://github.com/sgl-project/sglang/pull/26971
• @Zyann7 made their first contribution in https://github.com/sgl-project/sglang/pull/27122
• @DaZhUUU made their first contribution in https://github.com/sgl-project/sglang/pull/28043
• @prajjwal1 made their first contribution in https://github.com/sgl-project/sglang/pull/27588
• @cccccya made their first contribution in https://github.com/sgl-project/sglang/pull/28031
• @qinsir5522 made their first contribution in https://github.com/sgl-project/sglang/pull/28283
• @jinhaosong-source made their first contribution in https://github.com/sgl-project/sglang/pull/28004
• @feliang-git made their first contribution in https://github.com/sgl-project/sglang/pull/24515
• @jvmncs made their first contribution in https://github.com/sgl-project/sglang/pull/28002
• @Jyothirmaikottu made their first contribution in https://github.com/sgl-project/sglang/pull/28338
• @Qeeweew made their first contribution in https://github.com/sgl-project/sglang/pull/27328
• @okorzh-amd made their first contribution in https://github.com/sgl-project/sglang/pull/28486
• @stellaxcpeng made their first contribution in https://github.com/sgl-project/sglang/pull/28436
• @kangwangamd made their first contribution in https://github.com/sgl-project/sglang/pull/27815
• @lmyybh made their first contribution in https://github.com/sgl-project/sglang/pull/27553
• @joerowell made their first contribution in https://github.com/sgl-project/sglang/pull/28400
• @Talantan1102 made their first contribution in https://github.com/sgl-project/sglang/pull/25144
• @ashishdatta made their first contribution in https://github.com/sgl-project/sglang/pull/24082
• @jaybe1234 made their first contribution in https://github.com/sgl-project/sglang/pull/23910
• @shuwang21 made their first contribution in https://github.com/sgl-project/sglang/pull/28665
• @liyucheng09 made their first contribution in https://github.com/sgl-project/sglang/pull/26312
• @Zhangpch2021 made their first contribution in https://github.com/sgl-project/sglang/pull/23377
• @VarV0id made their first contribution in https://github.com/sgl-project/sglang/pull/26773
• @jeremyzhang866 made their first contribution in https://github.com/sgl-project/sglang/pull/26923
• @1e4ves made their first contribution in https://github.com/sgl-project/sglang/pull/28619
• @shihaoustc made their first contribution in https://github.com/sgl-project/sglang/pull/28718
• @pjdurden made their first contribution in https://github.com/sgl-project/sglang/pull/27430
• @EazyReal made their first contribution in https://github.com/sgl-project/sglang/pull/28802
• @Terry-Uv made their first contribution in https://github.com/sgl-project/sglang/pull/26880
• @yokinoshitayoki made their first contribution in https://github.com/sgl-project/sglang/pull/26574

Full Changelog: https://github.com/sgl-project/sglang/compare/v0.5.13...v0.5.14

Highlights

New Model Support:

• Autoregressive: Nemotron 3 Ultra (Day-0, blog), Step-3.7-Flash, Command A+
• Diffusion: Cosmos3, LingBot-World, SANA-WM, Ernie-Image, FLUX.2-Klein 4B/9B, Ideogram 4

Spec V2 is now the default speculative-decoding path: Tree drafting with topk > 1 is production-ready across the triton / FA3 / MLA / aiter backends, including page_size > 1 and Mamba/hybrid-linear models (#26997, #26972, #27463). Spec V1 is deprecated, with EAGLE/MTP now running on the unified V2 worker (#25464), and topk = 1 drafting is faster (#26397, #26424).

Lower per-step scheduler overhead: Unified async value passing through FutureMap plus moving prefill input transfer onto the forward stream reduced per-step launch overhead and improved stability under high concurrency (#25945, #25879, #26380).

Piecewise & Breakable CUDA Graph coverage: Piecewise (PCG) and Breakable (BCG) CUDA Graph capture more of the model to cut per-step kernel-launch overhead, now extended to DSA models, Kimi-K2.5, and DeepSeek V4: #23351, #26382, #25195.

Faster Qwen 3.5 on Blackwell: New FlashInfer Gated DeltaNet (GDN) kernels and a CuTeDSL GDN prefill kernel speed up Qwen 3.5 on Blackwell GPUs: #22921, #23273, #26200.

HiCache for hybrid models by default: HybridModel (SWA/Mamba) launches HiCache through UnifiedTree by default, bringing hierarchical KV-cache offload to sliding-window and Mamba hybrids out of the box: #27759.

Heterogeneous CPU + GPU EPD disaggregation (with Intel): Offload VLM vision encoding onto Intel Xeon CPUs alongside GPUs, with up to ~1.3x P99 TTFT and request-throughput gains under load. (blog)

MoRI on AMD Instinct MI355X (with AMD): Cost-competitive DeepSeek-R1 disaggregated inference via AMD's MoRI communication library, $0.169 per million tokens at 129 tok/s/user. (blog)

DeepSeek V4 — context parallelism & sparse-attention kernels: Building on the v0.5.12 Day-0 path, v0.5.13 extends DeepSeek-V4 to context-parallel serving and adds its sparse-attention kernels:

• Context Parallel + MTP: #24934
• Context Parallel + fused MoE kernel (non-DeepEP): #24947
• Sparse FlashMLA via flash_mla_sparse_fwd: #25418
• FP4 indexer support: #26209
• SM120 support: #24692
• DeepEP waterfill load balancing: #25391
• MHC kernel warmup: #25810
• Breakable CUDA Graph for DeepSeek V4: #25195
• Backed by sgl-kernel 0.4.3 exposing sgl_kernel.flashmla: #26421, #26132

See the DeepSeek-V4 cookbook for tuned deployment commands.

SGLang-Diffusion — realtime & progressive resolution: OpenAI-style realtime video generation with msgpack frame streaming and a standalone browser WebUI (#26954, #26959), continuous camera controls + super-resolution controls (#27026, #27297), and progressive-resolution growing across FLUX / FLUX.2 / Qwen-Image / Wan / Z-Image (#27524).

---

Full release notes by category below.

New Model Support

• Nemotron 3 Ultra (Day-0, kernel optimizations): #26733 (see cookbook)
• Step-3.7-Flash: #26565 (see cookbook)
• Cosmos3-Nano / Cosmos3-Super — T2V / I2V / T2I (Diffusion): #24994, #26492, #26926, #26950 (see cookbook)
• Ernie-Image (Diffusion): #22439, #27195 (see cookbook)
• LingBot-World — realtime / causal-DMD generation (Diffusion): #26954 (see cookbook)
• SANA-WM — streaming + realtime (Diffusion): #27531 (see cookbook)
• FLUX.2-Klein 4B / 9B (Diffusion): #25661 (see cookbook)
• Ideogram 4 — FP8 / NVFP4 with tensor parallelism (Diffusion): #27279, #27379, #27393 (see cookbook)
• Command A+ (Cohere 2 family): #26106, #27401 (cookbook page pending)

DeepSeek V4

• Context Parallel + MTP: #24934
• Context Parallel + fused MoE kernel (non-DeepEP): #24947
• MHC kernel warmup: #25810
• SM120 support: #24692
• FP4 indexer support: #26209
• Integrate flash_mla_sparse_fwd kernel: #25418
• DeepEP waterfill support: #25391
• Breakable CUDA Graph for DeepSeek V4: #25195

Speculative Decoding

• Spec V2 is now the default speculative-decoding path
• Tree speculative drafting (topk > 1) on Spec V2 — page_size > 1 and Mamba/hybrid-linear, validated across triton / FA3 / MLA / aiter: #26997, #26972, #27463
• Spec V1 deprecated; EAGLE/MTP run on the unified V2 worker: #25464
• Spec V2 extended to the FlashMLA backend: #24640
• Adaptive speculative decoding: batch-size-aware num_steps + observability metrics: #24055, #25940
• Faster topk = 1 drafting (skip full-vocab softmax + redundant cat/topk/sort/gather ops): #26397, #26424
• Draft-extend CUDA Graph for the trtllm mha attention backend: #25002

Piecewise & Breakable CUDA Graph

• PCG support for DSA models: #23351
• PCG support for Kimi-K2.5: #26382
• BCG support for DeepSeek V4: #25195

Context Parallelism

• Support bs > 1 for prefill CP: #23269
• Prefill CP for MLA models (Kimi K2.5, DeepSeek V3): #23292

Attention Backends

• CuTeDSL MLA attention kernels from FlashInfer: #24737
• Qwen 3.5: FlashInfer GDN kernels on Blackwell: #22921, #23273
• Qwen 3.5: CuTeDSL GDN prefill kernel on Blackwell: #26200

Scheduler & Runtime

• Overlap Scheduler: fewer CPU–GPU sync points
• Unified async value passing through FutureMap; prefill input transfer moved onto the forward stream, reducing per-step launch overhead and improving stability under high concurrency: #25945, #25879, #26380
• Runtime memory-safety checks enabled by default in CI to guard correctness continuously: #27461, #26335

HiCache & Radix Cache

• HybridModel (SWA/Mamba) launches HiCache via UnifiedTree by default: #27759

PD Disaggregation

• Optimistic prefill for better TTFT: #26780
• Decode-side HiCache integration for incremental KV-cache transfer: #26227
• HiSparse support for DeepSeek V4 with PD: #24880
• Notify and cancel KV-cache transfer for aborted requests: #27372
• Pipeline Parallelism (PP) + PD support for DeepSeek-V4: #24704
• EPD disaggregation support for MiMo-V2: #24931
• Tolerate KV pools without end_layer (Qwen3-Next disagg): #25476
• Unstick decode aborts under prealloc pressure: #25561
• Un-blacklist mooncake sessions when a probe succeeds: #25287
• HiSparse + PD: support host memory-pool page > 1: #23606
• Support regular worker discovery alongside PD workers in IGW mode: #25294

LoRA

• Experimental fast LoRA path with experimental_sgl_trtllm MoE backend for FP8 and NVFP4 models: #27329
• Remove synchronous .any().item() guard in the LoRA MoE prefill path: #25531
• Share MoE LoRA info: #24160
• More efficient pinned memory: #20876
• Fix overlap loading for cancelled requests: #25413
• Fix LoRA pool not appearing in /v1/loads: #25440

Multimodal

• Gemma 4: encoder-free variant unifying text, vision, and audio in one model: #27167 (see cookbook)

SGLang-Diffusion

Realtime diffusion

• OpenAI-style realtime video generation: #26954
• Msgpack frame streaming + standalone browser WebUI: #26959
• Continuous camera controls + super-resolution controls: #27026, #27297
• Lossless RGB transport improvements: #27236

Progressive resolution

• Progressive-resolution growing for image and video generation (FLUX, FLUX.2, Qwen Image, Wan, Z-Image): #27524

Memory & residency

• Layerwise offload generalized beyond legacy DiT components: #24593
• Memory-aware component load order: #25457
• Encoder layerwise-offload defaults: #25517
• Role-based component loading + stage affinity: #25168
• Combined DiT CPU offload + layerwise offload: #26925

Quantization & backends

• Ideogram 4 FP8 / NVFP4 support: #27279, #27379
• FlashInfer TRTLLM as the default diffusion NVFP4 backend: #25523
• Wan2.2 ModelOpt checkpoint updates: #25483, #25857

Performance

• Cosmos3 serve / denoising / I2V / parallel-decode / fused QKNorm optimizations: #26926, #26973, #27037, #27041, #27084, #27096
• LingBot realtime transport / SP cache path / camera-conditioning optimizations: #27023, #27297, #27383
• USP varlen-FA fast path + replicated-KV prefix all-to-all optimization: #26318, #27143
• UniPC scheduler GPU-sync removal: #27440

AMD / ROCm

• [AMD][DSV4] DSV4 MTP graph + sparse triton attention optimizations: #26383
• [AMD] DSV4 compressor optimization: #26208
• [AMD] Enable shared-experts fusion with the new Kimi-K2.5-MXFP4 model: #25390
• [AMD][aiter] Fix cuda_graph_kv_indices OOB under page_size > 1: #24587
• [AMD] Upgrade AITER: #25896

NPU / Ascend

• [NPU] Support chunked prefill for Qwen3.5 / Qwen3.6 models: #25839
• [NPU] Use Triton split_qkvgate_gemma_rmsnorm_rope for Qwen3.5 and Qwen3-Next: #23925
• [NPU] Support DeepSeek-OCR and DeepSeek-OCR-2: #25257
• [Diffusion][NPU] Add attention backends for diffusion models on Ascend NPU: #23482
• [Diffusion][NPU][Quant] Add MXFP4 quantization support for Wan2.2 on Ascend NPU: #22338
• [Diffusion][NPU] Disaggregation diffusion-stage support for NPU: #25895

CPU / Intel / MUSA / MLX

• [MLX] Support Qwen3.5 (dense) model: #25754
• [CPU] Add support for Qwen3-VL and Qwen3-Omni: #12662
• [CPU] Add GPT-OSS model optimization for CPU: #16775
• [CPU] Faster KV-cache writes: #25874
• [CPU] Explicitly enable AVX512 & AMX instruction sets: #26145
• [Xeon] CPU CI enhancement for Intel Xeon platforms: #24649
• [MUSA][Diffusion] Improve Wan model inference speed using torch.compile: #25256

Dependencies

• transformers 5.6.0 → 5.8.1: #25451
• flashinfer 0.6.11.post1 → 0.6.12: #26854
• xgrammar 0.2.0 → 0.2.1: #25676
• sgl-kernel 0.4.2.post2 → 0.4.3 (sgl_kernel.flashmla + DeepSeek V4 kernels): #26421, #26132
• nvidia-cutlass-dsl 4.5.1 → 4.5.2: #26854

Security

No security-tagged PRs in this release.

---

All PRs included in this release: https://github.com/sgl-project/sglang/compare/v0.5.12...v0.5.13

New Contributors

• @jasonjk-park made their first contribution in https://github.com/sgl-project/sglang/pull/24999
• @JoeLee314 made their first contribution in https://github.com/sgl-project/sglang/pull/25380
• @zhengluo-nv made their first contribution in https://github.com/sgl-project/sglang/pull/24723
• @yuychang made their first contribution in https://github.com/sgl-project/sglang/pull/25260
• @amd-bishwoadhikari made their first contribution in https://github.com/sgl-project/sglang/pull/22371
• @miamia0 made their first contribution in https://github.com/sgl-project/sglang/pull/25180
• @Xia-Weiwen made their first contribution in https://github.com/sgl-project/sglang/pull/21668
• @JINO-ROHIT made their first contribution in https://github.com/sgl-project/sglang/pull/25178
• @Gruner-atero made their first contribution in https://github.com/sgl-project/sglang/pull/25293
• @EanWang211123 made their first contribution in https://github.com/sgl-project/sglang/pull/23331
• @nagisa-kunhah made their first contribution in https://github.com/sgl-project/sglang/pull/24640
• @kflansburg made their first contribution in https://github.com/sgl-project/sglang/pull/25287
• @xiaobao520123 made their first contribution in https://github.com/sgl-project/sglang/pull/25786
• @zjd0112 made their first contribution in https://github.com/sgl-project/sglang/pull/25770
• @jiayisunx made their first contribution in https://github.com/sgl-project/sglang/pull/25730
• @rllin made their first contribution in https://github.com/sgl-project/sglang/pull/25807
• @moehanabi made their first contribution in https://github.com/sgl-project/sglang/pull/21191
• @abinggo made their first contribution in https://github.com/sgl-project/sglang/pull/24751
• @alex0dd made their first contribution in https://github.com/sgl-project/sglang/pull/25661
• @FredHuang99 made their first contribution in https://github.com/sgl-project/sglang/pull/25168
• @zhangtao2-1 made their first contribution in https://github.com/sgl-project/sglang/pull/25600
• @longxin9715 made their first contribution in https://github.com/sgl-project/sglang/pull/26069
• @nadongjun made their first contribution in https://github.com/sgl-project/sglang/pull/24610
• @hanwlax made their first contribution in https://github.com/sgl-project/sglang/pull/25856
• @vuuihc made their first contribution in https://github.com/sgl-project/sglang/pull/26177
• @yuhuiaws made their first contribution in https://github.com/sgl-project/sglang/pull/25404
• @ckvermaAI made their first contribution in https://github.com/sgl-project/sglang/pull/23757
• @chengchao23 made their first contribution in https://github.com/sgl-project/sglang/pull/19493
• @0-693 made their first contribution in https://github.com/sgl-project/sglang/pull/21544
• @vguduruTT made their first contribution in https://github.com/sgl-project/sglang/pull/22627
• @jbschlosser made their first contribution in https://github.com/sgl-project/sglang/pull/25911
• @nv-dmajchrowski made their first contribution in https://github.com/sgl-project/sglang/pull/24994
• @jvzibro made their first contribution in https://github.com/sgl-project/sglang/pull/26470
• @Shaoting-Feng made their first contribution in https://github.com/sgl-project/sglang/pull/24089
• @yao-matrix made their first contribution in https://github.com/sgl-project/sglang/pull/25174
• @hippothewild made their first contribution in https://github.com/sgl-project/sglang/pull/25973
• @cquil11 made their first contribution in https://github.com/sgl-project/sglang/pull/26590
• @jmamou made their first contribution in https://github.com/sgl-project/sglang/pull/24149
• @SKRohit made their first contribution in https://github.com/sgl-project/sglang/pull/26257
• @LucQueen made their first contribution in https://github.com/sgl-project/sglang/pull/22587
• @whn09 made their first contribution in https://github.com/sgl-project/sglang/pull/25083
• @BangBOOM made their first contribution in https://github.com/sgl-project/sglang/pull/26672
• @fcczzz made their first contribution in https://github.com/sgl-project/sglang/pull/25880
• @rbrugaro-amd made their first contribution in https://github.com/sgl-project/sglang/pull/25463
• @akhoroshev made their first contribution in https://github.com/sgl-project/sglang/pull/26673
• @KaisennHu made their first contribution in https://github.com/sgl-project/sglang/pull/24667
• @mpdfdfl made their first contribution in https://github.com/sgl-project/sglang/pull/26303
• @ntgiang71096 made their first contribution in https://github.com/sgl-project/sglang/pull/25669
• @decajoin made their first contribution in https://github.com/sgl-project/sglang/pull/26925
• @krishung5 made their first contribution in https://github.com/sgl-project/sglang/pull/25300
• @AliceChenyy made their first contribution in https://github.com/sgl-project/sglang/pull/24692
• @prakashkagitha made their first contribution in https://github.com/sgl-project/sglang/pull/25813
• @thanhhao98 made their first contribution in https://github.com/sgl-project/sglang/pull/26384
• @pengdurice made their first contribution in https://github.com/sgl-project/sglang/pull/26045
• @Li-brua made their first contribution in https://github.com/sgl-project/sglang/pull/27028
• @iyastreb made their first contribution in https://github.com/sgl-project/sglang/pull/26406
• @Ronnie-Rui made their first contribution in https://github.com/sgl-project/sglang/pull/27011
• @yeqcharlotte made their first contribution in https://github.com/sgl-project/sglang/pull/27085
• @JonnyKong made their first contribution in https://github.com/sgl-project/sglang/pull/26768
• @zhenghax made their first contribution in https://github.com/sgl-project/sglang/pull/26969
• @zx3xyy made their first contribution in https://github.com/sgl-project/sglang/pull/26746
• @ilia-iliev made their first contribution in https://github.com/sgl-project/sglang/pull/25292
• @bowenwan6 made their first contribution in https://github.com/sgl-project/sglang/pull/26864
• @akelch11 made their first contribution in https://github.com/sgl-project/sglang/pull/26859
• @gogongxt made their first contribution in https://github.com/sgl-project/sglang/pull/27374
• @256256mjw made their first contribution in https://github.com/sgl-project/sglang/pull/26444
• @Fatemanx made their first contribution in https://github.com/sgl-project/sglang/pull/27279
• @maodoudou168 made their first contribution in https://github.com/sgl-project/sglang/pull/24055
• @L4-1024 made their first contribution in https://github.com/sgl-project/sglang/pull/26356
• @Stella-17 made their first contribution in https://github.com/sgl-project/sglang/pull/27242

Full Changelog: https://github.com/sgl-project/sglang/compare/v0.5.12...v0.5.13

v0.5.12.post1 is a stability patch on top of v0.5.12. It cherry-picks 12 fixes — primarily for DeepSeek V4 — onto the release branch.

Bug Fixes

DeepSeek V4

• DSV4-Pro emits garbled text during single-token decode on B200/B300 (fix deep_gemm UE8M0 scale-packing path by ceiling activation scales before packing): #25733
• DSV4 + EAGLE/MTP in disaggregation decode crashes around 2000 requests with a SWA allocator assertion (recycled KV pages kept stale sliding-window mappings): #25805
• DSV4 NSA prefill context-parallel (--enable-nsa-prefill-context-parallel --nsa-prefill-cp-mode round-robin-split) in --disaggregation-mode prefill: scheduler crash at startup: #25396
• DSV4 HiSparse + SGLANG_OPT_USE_COMPRESSOR_V2=1: GSM8K accuracy restored from 0.825 → 0.960: #25646
• DSV4 PD disaggregation now works with pipeline parallelism > 1 (removed stale pp_size=1 assertion): #25771
• DSV4-Flash with --load-format dummy + FlashInfer mxfp4 hits CUDA illegal memory access during CUDA-graph capture (the integer HashTopK.tid2eid lookup table was left uninitialized by dummy load): #25892
• DSV4 HiCache + SGLANG_OPT_CACHE_SWA_TRANSLATION=1 returns stale translation indices after a cache rebuild, causing OOB writes / wrong outputs: #25889

Disaggregation

• [PD][NIXL] Always send aux on is_last; only expect state when truthy: #25699

Other

• Fix missing group arg in get_dp_buffer: #25585

Performance

• DSV4: warm MHC token-count buckets at startup (gated to SGLANG_OPT_DEEPGEMM_HC_PRENORM=1 + SGLANG_OPT_USE_TILELANG_MHC_PRE=1 + hybrid SWA) to eliminate 20–40s cold-bucket forward stalls: #25810
• DSV4-Pro: precompile a DeepGEMM branch for _dispatch_bf16_fp32_backend to cut runtime JIT compile cost: #25860

Dependencies

• Use [cu13] extra for nvidia-cutlass-dsl (default to CUDA 13; required for sm_103 / B300): #25576

All PRs included in this release: https://github.com/sgl-project/sglang/compare/v0.5.12...v0.5.12.post1

Full Changelog: https://github.com/sgl-project/sglang/compare/v0.5.12...v0.5.12.post1

Highlights

• DeepSeek V4 support: Full inference path for DeepSeek-V4 (#23882), including:

  Day-0 Features: #23882

  • Parallelism: Tensor Parallelism/Expert Parallelism/Context Parallelism/Data Parallel Attention
  • Hardware: Nvidia B300/B200/H200/H100/GB200/GB300, AMD MI35X
  • Prefill-Decode Disaggregation
  • HiSparse for offloading inactive KV cache to CPU memory
  • Reasoning parser and Tool Call Parser
  • DeepGemm and FlashMLA kernels for DeepSeek V4, including MegaMoE

  Post-Day-0 additions:

  • HiCache for DeepSeek V4 under unified Radix Tree [UnifiedTree]: #24691
  • W4A4 MegaMoE kernels — faster speed with negligible accuracy drop: #25052
  • Marlin/FlashInfer W4A8 MoE kernels on Hopper: #24816 #24986
  • Faster V2 fused compression kernels: #24890
  • TP16 support on H100/H20: #24949
  • Fused SiLU+clamp+FP8 quant kernel: #24897
  • Optimized MHC + DeepGemm pipeline (fused norm, fused hc_head): #24775
  • Non-standard chat template support for DSv4: #23915
  • Multi-detokenizer support: #24944
  • Pipeline Parallelism + PD support for DeepSeek-V4: #24700
  • A unified docker tag lmsysorg/sglang:v0.5.12 for all Nvidia GPUs

  See the LMSYS blog and the DeepSeek-V4 cookbook for more details.

• TokenSpeed MLA attention backend (Blackwell, FP8 KV cache): New MLA prefill/decode kernels integrated as an attention backend on SM100, with FP8 KV cache support for low-latency MLA serving: #24925

• DSv3.2 / GLM-5 FP4 low-latency perf: PDL enabled across DSv3.2 / GLM-5 kernels, torch.mm for the DeepSeek V3.2 indexer GEMM, and a reland of the Cute-DSL FP4 dense GEMM — materially trimming low-latency overheads on FP4 paths: #23965, #23856, #23590, #25311

• New Model Support: DeepSeek V4 #23882, Intern-S2-Preview #24875, MiniCPM-V 4.6 #24855, Laguna-XS.2 #24204, Ring-2.6-1T #25360, and Gemma 4 MTP #24436 — with cookbook recipes for tuned deployment commands. See docs.sglang.io/cookbook

• HiCache + UnifiedRadixTree: HiCache framework support for UnifiedRadixTree (with SWA), HiCache for DeepSeek V4, SSD offload through Mooncake store, and stability fixes across cascade eviction, tombstone replay, and partial-match paths: #23316, #23391, #24691, #24277, #24943, #24972, #25068, #25277

• Speculative Decoding V2 maturation: Adaptive Spec V2, EAGLE-3 SWA + newer drafters, Kimi K2.5 EAGLE-3 MLA, Gemma 3/4 + EAGLE-3, and an extensive naming / shape-handling refactor across draft-extend paths: #23336, #24663, #24664, #24826, #23976, #24859

• CUDA 13 DeepEP migration: Gateway DeepEP source swapped from a community fork to deepseek-ai/DeepEP@hybrid-ep so DeepEP builds and runs cleanly on the CUDA 13 default; FlashInfer pinned at 0.6.11.post1 alongside a gpt-oss triton-kernel fix: #25113

New Model Support

Entries with a published cookbook recipe come first; entries whose cookbook page is still pending are grouped at the bottom.

• DeepSeek V4 (see cookbook; LMSYS blog)
• Intern-S2-Preview: #24875, #25115, #25134 (see cookbook)
• MiniCPM-V 4.6: #24855, #24876, #24991, #24998 (see cookbook)
• Laguna-XS.2 (Poolside): #24204, #24730 (see cookbook)
• Ring-2.6-1T (InclusionAI, trillion-param reasoning): #25360, #25370 (see cookbook)
• Gemma 4 MTP (MTP head for Gemma 4): #24436, #24433
• Trinity-mini (Ascend NPU, ~90% accuracy): #18172
• HunyuanVideo ModelOpt FP8 (Diffusion): #23199
• Qwen Image ModelOpt FP8 (Diffusion): #23155

Speculative Decoding

• TokenSpeed MLA prefill/decode kernels integrated as attention backend (FP8 KV cache, Blackwell): #24925
• Adaptive Spec V2 (2/N): #23336
• SWA support for EAGLE-3 drafter: #24664
• Support newer EAGLE-3 drafters: #24663
• Kimi K2.5 EAGLE-3 MLA spec decoding: #24826
• Gemma 3 / Gemma 4 + EAGLE-3 support: #23976
• Spec V1 — split draft-extend into EagleDraftExtendInput: #24859
• Custom speculative-algorithm registry: #23991
• Spec-V2 overlap stale-state fix: #23456
• trtllm decode kernel for draft extend: #24566
• AMD: EAGLE on Qwen3.5 FP8/MXFP4 via aiter unified attention: #23146
• Fix Kimi K2.5 MLA EAGLE + DP attention: #25033
• Fix ngram metric off-by-1 in num_accepted_drafts_per_req_cpu: #24965
• Fix frozen-KV MTP crash when bonus_tokens is None: #25204
• Fix stuck-MTP on DSA models: #24635
• Reduce specdec CPU overhead: #23321
• Spec-decoding naming-convention rule + refactors: #24094, #25014, #25038, #24081, #24724, #24735, #24881, #25010, #25012, #25030, #25029, #25037, #25109

PD Disaggregation

• DSv4 Flash disaggregation test: #24973
• Unify DSv4 dispatch with SWA: #24888
• DSv4 mooncake state_type branch: #24878
• Hybrid state transfer refactor: #24932
• Priority scheduling in PD mode fix: #25062
• NIXL: staging buffer for heterogeneous-TP KV transfer: #22536
• NIXL: async transfer: #23967
• NIXL XPU: uint64 pointer overflow + mismatched P/D TP fixes: #24188, #24648
• Mooncake: incremental transfer + SSD offload: #24257, #24277
• Multi-node prefill bootstrap-port broadcast: #24378
• Add retry-with-backoff for prefill bootstrap registration: #25125
• PrefillDelayer: NCCL all-gather for cross-DP info sync: #24768
• MORI-IO: state transfer + high-concurrency fixes: #22665
• Per-room cleanup centralization; prevent update_status from cleared entries; fix abort update_status across KV backends: #24601, #24539, #24522
• PD KV transfer metrics fix: #24416
• SWA memory preallocation for disaggregated decode: #24857
• IntraNode NVLink configuration docs: #23329

HiCache & Radix Cache

• HiCache framework for UnifiedRadixTree: #23316
• SWA HiCache for unified radix cache: #23391
• HiCache for DeepSeek V4 + nightly CI for DSA model: #24691, #25369, #25348
• SSD offload through Mooncake store: #24277
• HiSparse FP8 KV cache via flashmla_kv backend: #23013
• Default storage prefetch timeout: #23309
• UnifiedRadixCache device match semantics with HiCache: #25277
• UnifiedTree partial match on evicted+backuped nodes: #24943
• UnifiedTree tombstone lock release replay fix: #24972
• UnifiedTree _cascade_evict leaf determination fix: #25068
• UnifiedRadixTree align cache_empty_result with RadixTree: #24779
• Mamba radix cache KV events; SWA radix cache events: #23678, #24718
• SWA chunk req deferred fix; SWA component host hit fix: #24318, #25085

LoRA

• MLA attention LoRA (q_b_proj / kv_b_proj): #25001
• CSGMV backend with virtual experts for MoE LoRA: #24007
• MoE LoRA: remove CPU-GPU sync barriers and duplicate code (prefill optimize 2/n, 3/n): #24246, #24262
• LoRADrainer for high P99 TTFT: #17913
• qkv_proj buffer sizing when tp_size > num_key_value_heads: #24420
• Torch-Native LoRA: embedding + graph optimization: #21885
• Deterministic lora_id for multi-node --lora-paths: #24555
• Fix broken sgemm_lora_a_graph_fwd due to invalid torch.mm(): #24760
• Diffusion: fix RowParallel LoRA merged forwarding: #24410

Performance

• TMA bulk-store set_mla_kv_buffer (up to 12× over baseline): #25311
• Kimi tokenizer TTFT optimization: #25265
• Avoid hidden-states D2H copy when return_hidden_states=false: #25155
• DeepseekV2MoE: defer shared experts when routed kernel is non-mutating: #25279
• SGLANG_OPT_FP8_WO_A_GEMM on by default: #25181
• --prefill-only-disable-kv-cache to skip KV pool allocation: #23675
• Gemma 4 MoE: fused Q/K/V RMSNorm + per-expert FP8 ckpt loader: #24696
• Gemma 4 VLM: PCG + fused RMSNorm + residual: #24048
• MHC pipeline: DeepGemm + fused norm + fused hc_head: #24775
• JIT custom all-reduce default; non-NVL follow-up: #24363, #24742
• SGLANG_USE_JIT_ALL_REDUCE → SGLANG_OPT_USE_CUSTOM_ALL_REDUCE_V2: #24297
• Eliminate logits H2D blocking copy: #24627
• Cache empty MatchResult in RadixCache: #24470
• Breakable CUDA graph for bs > 1: #24662
• FA3: skip scheduler_metadata precompute under DP attention: #24632
• aten::rms_norm / aten::mm.dtype registration in batch-invariant mode: #24459
• Optimize Helios fused norm modulation: #24059
• Z-Image packed QKV optimization: #24117
• KDA prefill kernels: diagonal + recompute fuse: #24271

Observability

• sglang:get_loads_duration_seconds Prometheus metric: #25163
• Per-iteration forward-pass metrics via ZMQ PUB: #22789
• SGLANG_TRACE_LEVEL env for startup trace level: #24716
• fwd_occupancy metric in SchedulerStats + Prometheus collector: #24458
• SWA / Mamba cache metrics: #24396
• Mamba radix cache + SWA radix cache KV events: #23678, #24718
• PD KV transfer metrics fix: #24416
• CP allgather buffer registered with symmetric memory: #24040
• Decode-side bootstrap/alloc metrics + non-int token-id filter: #24684

Frontend & API

• /v1/tokenize chat-completion-style support: #23981
• Multi-detokenizer support: #24944
• Structural tags for strict tool calling & reasoning across more models: #21722
• Auto-detect reasoning / tool-call parser from chat template: #23952
• Two-phase reasoning grammar + --enable-strict-thinking: #23953
• OpenAI reasoning.enabled mapping to thinking + enable_thinking: #23951
• Kimi-K2.5 bare-numeric tool-call IDs: #23950
• Crusoe managed-inference backend: #20475
• Azure Blob Storage connector (az:// and *.blob.core.windows.net): #23995
• Adaptive queue-based prefill-delayer trigger: #23189
• SGLANG_MAX_KV_CHUNK_CAPACITY env: #25120
• SGLANG_RADIX_FORCE_MISS env: #24726, #24950
• Reject repetition_penalty=0 in SamplingParams.verify(): #24874
• --random-input-len for send_one.py: #24464

SGLang-Diffusion

• New model support: HunyuanVideo ModelOpt FP8 (#23199), Qwen Image ModelOpt FP8 (#23155)
• CFG parallelism framework + multi-branch CFG for LTX-2: #23736
• Initial dynamic batching: #18764
• Performance-mode server args: #24491
• dit_precision config respected (no hardcoded bf16): #24988
• Cache-DiT: mount before torch.compile in native denoising: #25328
• Z-Image Cache-DiT sequence-parallel override fix: #25305
• USP: direct all-to-all collectives; NCCL deadlock fix for remainder seq lengths: #24366, #24694
• FA3 varlen out argument handling: #24688
• RowParallel LoRA merged forwarding fix: #24410
• CFG communication: handle non-contiguous tensors: #24332
• LTX-2.3 alignment with official + HQ denoising split passes: #24313, #24298
• LTX-2 feed-forward TP optimization (#23221) + Hunyuan3D shape denoising / export chunks: #24287, #24358
• Encoder result cache for default negative prompt: #24304
• Channels-last 3D VAE convs by default; disable VAE CPU offload by default: #23200, #24315
• Component attention-backend override CLI: #24320
• AMD: online MXFP4 + FP8 diffusion quantization; aiter RMSNorm; temporal-unfolded batched Conv2D for ROCm VAE decode; dual-stream MoE: #21431, #24360, #22971, #24005, #24677
• NPU: MXFP8 quantization for Wan2.2 (#20922, #24918); fused-operator E2E perf for Wan (#24028); selectable parallel VAE decode strategies (#23248); SANA fix (#24798); Z-Image negative-branch rotary embed CFG fix (#23538)
• MUSA: sage attention backend (#24752)

AMD / ROCm

• DSv4 Flash / Pro nightly tests on MI35x ROCm 7.2: #24203, #24825, #25039
• NSA indexer fallbacks + preshuffle paged MQA + GLM-5 NSA TileLang: #24125, #23562, #25205
• fp8 blockwise quantization combine for MoRI EP: #24879
• gfx950 + aiter _skip_rope_for_aiter_fused_mla: #24148
• aiter fused_qk_rmsnorm API shim (pre/post #2958): #24799
• TBO Spec-V2 seq_lens_cpu None handling: #24319
• Kimi-K2.6 nightly tests (MI30x / MI35x): #23848
• JIT kernel PR-CI through run_suite.py: #24987
• AMD JIT benches: clamp position + resolve-token-ids: #24209, #24210 (#25209, #25210)
• AMD CI hygiene (registration + cleanup + VRAM): #24569, #24572, #24586, #24612, #24614, #24615, #24665, #24924, #24981, #25112
• Docker: cache-dit 1.3.0 pin; archive.ubuntu.com fallback: #24924, #24407

NPU / Ascend

• zbal support: #24575
• Trinity-mini support (~90% accuracy): #18172
• Shared-expert dual-stream optimization: #23827
• Mamba-extra-buffer radix cache (Qwen3.5): #23891
• MLA KV transfer in pipeline parallel: #23893
• Multi-batch FIA ops: #20177
• GLM-5 docs: DeepEP enabled by default: #23708
• GLM-4.5V / GLM-4.7-Flash NPU support / fixes (carry-over): existing
• --disable-cuda-graph + MTP warmup fix: #23819
• MRoPE position fix in Eagle Worker v2 with PlanStream: #23423
• Z-Image negative-branch rotary embeddings for CFG: #23538
• Wan quantization fix: #24540
• causal_conv1d_update_v2 for performance: #24595
• sgl-kernel-npu 2026.05.01 bump: #24951
• Profiler revert + re-add: #24685, #24815
• Doc / accuracy / FAQ work: #21537, #24658, #24676, #24777, #25114, #25130, #25268, #24668, #24918

CPU / Intel / MUSA / MLX / Apple Silicon

• MUSA: FlashInfer sampling backend: #24978
• MUSA: optimized kernels for piecewise CUDA graph: #23633
• MUSA: optimized kernels for hot ops: #23255
• MUSA: torchada 0.1.54 bump: #24592
• MLX: on-the-fly --quantization mlx_q4 / mlx_q8 on Apple Silicon: #24907
• MLX: auto-detect MLX-format quantization_config dict: #25191
• MLX: thread --quantization through MlxModelRunner in bench_one_batch: #25221
• MLX: Apple Silicon Metal kernel support in sgl-kernel: #23449
• sgl-kernel/cpu: w8a8 int8 model support for arm cpu: #16045
• Intel CPU tests migrated to test/registered (re-applied after revert): #25139, #22670, #25044
• Arm64 CPU Phase-1A CI bootstrap: #22123
• XPU pipeline parallelism on Intel: #23472

Quantization & Kernels

• NVFP4 hot-reload-safe weight loading (alias-when-same-shape): #25190
• NVFP4: free unused source scales after weight processing: #25107
• Cute-DSL NVFP4 quantization kernels: #23745
• Cute-DSL FP4 dense GEMM (reland): #23590
• DSv3.2 indexer GEMM via torch.mm: #23856
• PDL for DSv3.2 / GLM-5 kernels: #23965
• DSv4: W4A4 MegaMoE; W4(MXFP4)A16 on Hopper: #25052, #24986
• FlashInfer SM90 cutlass MXFP4 MoE backend (W4A16) for GPT-OSS + DSv4: #24816
• Port KV Compression V2 + fused SiLU+clamp+FP8 quant from DSV4 dev branch: #24890, #24897
• BF16 EP-MoE for DeepGEMM: #17392
• DeepGEMM deprecated in sgl-kernel; custom sgl-deep-gemm wheel + release workflow: #24268, #24348, #24385
• TRT-LLM A2A dispatch: NaN sanitization in padding slots: #24850
• TRT-LLM BF16 MoE for MTP: #24260
• MegaMoE decoupled from DeepEP backend (subsequently reverted): #24884, #25317
• DeepEP waterfill load balancing for shared-expert dispatch: #19290
• DeepEP support for --enable-return-routed-experts: #16859

Dependencies

• FlashInfer 0.6.8.post1 → 0.6.11 → 0.6.11.post1 (with intermediate revert): #24452, #25129, #25310, #25335
• sgl-kernel 0.4.2.post1, 0.4.2.post2: #24457, #25326
• sgl-kernel: SM90 flashmla compile fix: #24130
• Custom sgl-deep-gemm wheel + release workflow: #24268, #24348, #24385
• sgl-kernel-build x86 + arm merged into reusable workflow; disk-reclaim cleanup: #25135, #25206
• DeepEP swapped from fzyzcjy fork to deepseek-ai/DeepEP@hybrid-ep (CUDA 13): #25113
• Torch 2.11 Docker prep + dependency cleanup: #23593
• nixl stub installation alongside nixl-cuXX binary: #24369
• aarch64 cubin handling + masked-failure fix: #24234
• H20 stage on CUDA 13: #24916
• CUDA-13 kernel installation docs: #24181, #24516
• FlashInfer autotune cache: #24156
• FlashInfer workspace OOM fix: #24172
• FlashInfer allreduce fusion disabled under deterministic inference: #24629
• trtllm allreduce fusion with PDL: #23765
• TRTLLM MHA routing fix for draft-extend: #24856
• torchcodec → soundfile WAV fallback for trailing metadata: #24185
• sgl-kernel-npu 2026.05.01: #24951

Security

No security-tagged PRs in this window.

All PRs included in this release: https://github.com/sgl-project/sglang/compare/v0.5.11...v0.5.12

New Contributors

• @Seven-Streams made their first contribution in https://github.com/sgl-project/sglang/pull/21722
• @stargazerZJ made their first contribution in https://github.com/sgl-project/sglang/pull/24344
• @Jianhong-Zhang made their first contribution in https://github.com/sgl-project/sglang/pull/24188
• @gh1595 made their first contribution in https://github.com/sgl-project/sglang/pull/24420
• @revanthreddy-hai made their first contribution in https://github.com/sgl-project/sglang/pull/24329
• @TallMessiWu made their first contribution in https://github.com/sgl-project/sglang/pull/20922
• @ranimandepudi made their first contribution in https://github.com/sgl-project/sglang/pull/22123
• @Joey-gvwal made their first contribution in https://github.com/sgl-project/sglang/pull/23255
• @fanxingran made their first contribution in https://github.com/sgl-project/sglang/pull/24129
• @xz-keg made their first contribution in https://github.com/sgl-project/sglang/pull/24604
• @zhongdaor-nv made their first contribution in https://github.com/sgl-project/sglang/pull/23678
• @chfeng-cs made their first contribution in https://github.com/sgl-project/sglang/pull/24434
• @sglang-npu-bot made their first contribution in https://github.com/sgl-project/sglang/pull/24815
• @brian030128 made their first contribution in https://github.com/sgl-project/sglang/pull/24217
• @tjdharamsi made their first contribution in https://github.com/sgl-project/sglang/pull/24871
• @sytianhe made their first contribution in https://github.com/sgl-project/sglang/pull/24716
• @Dogacel made their first contribution in https://github.com/sgl-project/sglang/pull/24663
• @tangcy98 made their first contribution in https://github.com/sgl-project/sglang/pull/24967
• @1pikachu made their first contribution in https://github.com/sgl-project/sglang/pull/22670
• @flutist made their first contribution in https://github.com/sgl-project/sglang/pull/24760
• @acheamponge made their first contribution in https://github.com/sgl-project/sglang/pull/20475
• @taegeonum made their first contribution in https://github.com/sgl-project/sglang/pull/25022
• @RulinJuice made their first contribution in https://github.com/sgl-project/sglang/pull/24874
• @lluki made their first contribution in https://github.com/sgl-project/sglang/pull/24671
• @Religious-J made their first contribution in https://github.com/sgl-project/sglang/pull/20930
• @ltcs11 made their first contribution in https://github.com/sgl-project/sglang/pull/24575
• @damahua made their first contribution in https://github.com/sgl-project/sglang/pull/24907
• @ziang663 made their first contribution in https://github.com/sgl-project/sglang/pull/25126
• @Emmanuel0612 made their first contribution in https://github.com/sgl-project/sglang/pull/25209
• @Jialin made their first contribution in https://github.com/sgl-project/sglang/pull/25234
• @jlee5814 made their first contribution in https://github.com/sgl-project/sglang/pull/25191
• @unseenmars made their first contribution in https://github.com/sgl-project/sglang/pull/24935
• @nano8259 made their first contribution in https://github.com/sgl-project/sglang/pull/25125
• @imp2002 made their first contribution in https://github.com/sgl-project/sglang/pull/24130
• @liuxianglong17 made their first contribution in https://github.com/sgl-project/sglang/pull/25080

Full Changelog: https://github.com/sgl-project/sglang/compare/v0.5.11...v0.5.12

Highlights

• CUDA 13 + Torch 2.11: Default CUDA version moves to 13.0 across SGLang, sgl-kernel, and Docker images, and PyTorch is upgraded from 2.9 to 2.11 — modernizing the build matrix and unlocking newer kernels: #21247, #24162, #24183, #23593 (tracking issue #21498)

• Speculative Decoding V2 by default: Spec V2 (with overlap scheduling to hide CPU overhead) is now the default, materially reducing per-step CPU cost for EAGLE/MTP/DFLASH paths: #21062

• Decode Radix Cache for PD Disaggregation: Decode-side prefix caching now works under prefill/decode disaggregation, recovering radix-cache hit rates and TTFT savings for long shared prefixes in disaggregated deployments: #19746

• Day-0 / New Model Support: Gemma 4, GLM-5.1, Qwen3.6, MiMo-V2.5 / V2.5-Pro, Ling-2.6-Flash, Mistral Medium 3.5, and Kimi-K2.6 — with cookbook recipes for tuned deployment commands. See docs.sglang.io/cookbook: #21952, #23808, #23811, #23851, #23947, #23486, #23394

• DFLASH Speculative Decoding: New high-throughput spec-decode kernel from the kernel community, expanded across model backends and AMD ROCm: #22077, #22358, #22342, #23553

• FA3 Kernels from the Kernel Community: Drop-in FA3 kernels contributed by the community, integrated alongside FA4 to give users a high-performance option that's easy to maintain: #20796

• LoRA support for DeepSeek-V3 and Kimi-K2: LoRA now works on the largest MLA-based MoE models, including DeepSeek-V3 MLA LoRA and Kimi K2 — enabling adapter-based fine-tuning of frontier-scale models: #22323, #22381

• Context Parallel (CP) Enhancements: All-reduce + RMSNorm fusion under CP for end-to-end speedups, plus support for moe_dp_size = 1 paired with arbitrary attention_cp_size so MoE and attention parallelism can be tuned independently: #21249, #22003

• FlashInfer CuteDSL MoE Runner Backend: New dedicated FlashInferCuteDslMoE layer for the standard FP4 MoE path, giving an additional high-performance fused-MoE option: #21339

New Model Support

Entries with a published cookbook recipe come first; entries whose cookbook page is still pending are grouped at the bottom.

• Gemma 4: #21952 (and follow-ups #22079, #24048, #22842; see cookbook)
• GLM-5.1: #22543, #23037 (see cookbook)
• Qwen3.6: #23486 (see cookbook)
• MiMo-V2.5 / MiMo-V2.5-Pro: #23808, #23811, #23851, #23945, #24118 (see cookbook)
• Ling-2.6-Flash: #23947 (see cookbook)
• Mistral Medium 3.5: see cookbook
• Kimi-K2.6: #23394, #23408 (see cookbook)
• Hunyuan v3 (Tencent, preview): #23533 (see cookbook)
• FLUX.1-dev ModelOpt NVFP4 (Diffusion): #22672 (see FLUX cookbook)
• FLUX.2-small-decoder (Diffusion): #22414 (see FLUX cookbook)
• Qwen Image ModelOpt FP8 (Diffusion): #23155 (see Qwen-Image cookbook)
• LTX-2.3 / LTX-2.3 two-stage / TI2V (Diffusion): #22182, #22667, #22869 (see LTX cookbook)
• Qwen3-ASR (chunk-based streaming): #22073, #22089
• Voxtral (Mistral speech-to-text): #21635
• Parakeet (NVIDIA Nemotron encoder): #23568
• Moss-VL: #23454
• SequenceClassification model architecture (powers the Score API): #22118
• Stable Diffusion 3 medium (Diffusion): #19225
• ERNIE-Image (Diffusion): #22439
• JoyAI-Image-Edit (Diffusion): #22625

Speculative Decoding

• DFLASH speculative decoding initial support: #22077
• DFLASH enabled across additional model backends: #22358
• DFLASH speculative decoding on AMD ROCm: #22342
• Spec V2 enabled by default with overlap scheduling: #21062
• Penalty support for Spec V2 overlap scheduling: #22049
• Adaptive speculative_num_steps for EAGLE topk=1: #21599
• Allow piecewise CUDA graph with speculative decoding: #22128
• Eagle3 / DFLASH aux hidden state capture during CUDA graph init fixed: #22836
• Split accept_length into num_accepted_drafts / num_accepted_tokens: #23962
• DFLASH speculative decoding documentation: #23553

PD Disaggregation

• Decode-side radix cache support: #19746
• Incremental transfer for Mooncake transfer engine: #24257
• Allow PrefillDelayer in disaggregated-prefill mode: #23588
• NIXL: heterogeneous TP KV transfer for non-MLA models (Step 1/2 for Qwen3.5): #22145
• NIXL: Mamba state slice transfer for heterogeneous TP (Step 2/2 for Qwen3.5): #22240
• Bug fixes for IntraNode NVLink, MTP-layer KV transfer, and disagg-prefill DP rank resolution: #23252, #23539, #22901, #22990

Context Parallel & Parallelism

• All-reduce fusion support under CP: #21249
• moe_dp_size = 1 paired with arbitrary attention_cp_size: #22003
• All-reduce fusion enabled for DSA models: #22390
• Replace all-reduce + dp_scatter with reduce_scatterv for DP attention: #22642
• Step3p5: optimize all-reduce in MoE layers: #22773
• Pipeline parallelism on Intel XPU: #23472
• OpenTelemetry tracing for pipeline parallelism: #23169

LoRA

• DeepSeek-V3 MLA LoRA support and quantization-info refactor: #22323
• Kimi K2 LoRA support: #22381
• LoRADrainer to address high P99 TTFT: #17913
• Decoupled LoRA MoE backend with Marlin support: #21858
• Virtual experts for LoRA MoE (1/n): #22122, #24007
• CSGMV kernel offline auto-tuning: #20391
• Triton sgemm speedup with better grid selection: #22386
• Dual MoE CUDA graph capture for lora/nolora batches: #22809

Performance

• FA3 kernels from the kernel community: #20796
• Precompute FA3 scheduler_metadata to eliminate per-layer prepare cost: #21104
• Precompute gemma_weight to avoid redundant add on every forward: #22673
• Eliminate attention DtoD copy by passing pre-allocated output to FA: #21985
• Skip KV cache in FA backend for embedding mode: #21971
• O(1) RadixKey view for EAGLE bigram key: #23106
• PCG inductor path optimization for FP8 models: #23227
• Combo-kernels for horizontal fusion: #21977
• Optimize Gemma4 VLM with PCG and fused RMSNorm + residual add + scalar: #24048
• Restore torch.compile fusion for topk postprocessing: #21771
• Reduce unnecessary kernels and copies in the NSA indexer: #22232

Observability

• Pending token count surfaced in prefill log and get_load: #22480
• OpenTelemetry tracing for speculative decoding: #19545
• OpenTelemetry tracing for pipeline parallelism: #23169
• OpenTelemetry tracing in DiffGenerator: #21254
• Prometheus metrics endpoint for gRPC mode: #20801
• HTTP sidecar endpoints and FlushCache gRPC RPC for gRPC mode: #22500
• Raw KV cache pool token counts as Prometheus gauges: #22726

SGLang-Diffusion

• New model support: LTX-2.3 (#22182, #22667, #22869), ERNIE-Image (#22439), FLUX.2-small-decoder (#22414), JoyAI-Image-Edit (#22625), FLUX.1-dev ModelOpt NVFP4 (#22672), Qwen Image ModelOpt FP8 (#23155), Stable Diffusion 3 medium (#19225)
• ModelOpt diffusion FP8 support for Flux1/Flux2 and Wan2.2: #22365
• Standalone Rollout API + Denoising Environment Backpass + SP-Aligned Log-Prob for T2I post-training: #22604
• Disaggregated diffusion: #21701
• Dynamic batching v0: #18764
• CPU platform support for SGLang Diffusion: #20816
• AITER backends in Flux 2 pipeline (AMD): #22802
• LTX-2 feed-forward tensor parallelism optimization: #23221
• In-memory loading for URL/base64 image inputs (default): #23118
• Mixed-resolution benchmark support: #20863
• Auto-enable best parallel setting if unspecified: #22763

AMD

• MiniMax-M2.5 optimizations (aiter biased grouped topk; fused FP8 KV cache write): #23611, #23620
• Fused QK Gemma norm kernels (4 → fewer kernels): #23575
• Fused all-reduce + RMSNorm simplification: #21986
• GLM-5 / GLM-5.1 MXFP4 nightly accuracy + perf benchmarks (MI30x / MI35x): #21773, #22336
• MTP for GLM-5-mxfp4: #23219
• Aiter v0.1.12.post1 upgrade: #22264
• DFLASH speculative decoding enabled on ROCm: #22342
• Fix --page-size > 1 memory access fault with speculative decoding: #23596

NPU / Ascend

• Ascend backend supports Qwen3 MoE attention CP: #21685
• GLM-4.5V and GLM-4.7-Flash NPU support / fixes: #22961, #22509
• MTP for Qwen3.5: #20918
• TP communications compression for Qwen3 on NPU: #20520
• Add support-new-models documentation for NPU: #23824
• GGUF quantization for Ascend NPU (dense + MoE): #17883

CPU

• GPTQ / AWQ 4-bit quantization on CPU: #22685
• gemma4_rmsnorm_cpu kernel: #22842
• Qwen3.5 model optimization for CPU: #19484
• Apply routed scaling factor on output for biased grouped topk fusion: #22413
• Fix extend_attention_cpu / flash_attn_varlen_func NaN for large seq: #22434

Quantization

• MXFP4 quantized dense models on AMD CDNA2/CDNA3 GPUs: #19143 (later reverted in #23031, follow-up forthcoming)
• NVFP4 KV cache: quantization strategy abstraction and kernel: #21954
• DeepSeek-R1-0528-w4a8 + DeepEP Low-Latency FP8 dispatch: #22316
• MXFP8 sm100 path cleanup: #21881
• GLM-5/5.1 MXFP4 checkpoint inference compatibility fix: #22543

Dependencies

• Torch upgraded 2.9 → 2.11: #21247
• Default CUDA bumped to 13.0 across sglang, sgl-kernel, and Docker images: #21498 (tracking), #24162, #24183, #23593, #23119
• Flashinfer 0.6.7.post2 → 0.6.8.post1: #23281
• sgl-kernel bumped to 0.4.1.post1: #23720, #23733
• sgl-kernel bumped to 0.4.2: #24170
• Aiter v0.1.12.post1 (AMD): #22264

Security

• Fix for CVE-2026-5760: #23660
• Fix Trivy CVEs and cubin download 403s in Docker image: #22322

All PRs included in this release: https://github.com/sgl-project/sglang/compare/v0.5.10.post1...v0.5.11

New Contributors

• @AethoceSora made their first contribution in https://github.com/sgl-project/sglang/pull/23426
• @AlbeeSo made their first contribution in https://github.com/sgl-project/sglang/pull/23710
• @alec-flowers made their first contribution in https://github.com/sgl-project/sglang/pull/24090
• @AlonKejzman made their first contribution in https://github.com/sgl-project/sglang/pull/23753
• @amacaskill made their first contribution in https://github.com/sgl-project/sglang/pull/22537
• @AndyLi429 made their first contribution in https://github.com/sgl-project/sglang/pull/21685
• @Baichuan7 made their first contribution in https://github.com/sgl-project/sglang/pull/23060
• @ccullen-cert made their first contribution in https://github.com/sgl-project/sglang/pull/23660
• @ChangLiu0709 made their first contribution in https://github.com/sgl-project/sglang/pull/22908
• @charlotte12l made their first contribution in https://github.com/sgl-project/sglang/pull/21983
• @chenkaiyue made their first contribution in https://github.com/sgl-project/sglang/pull/17195
• @chx96642264 made their first contribution in https://github.com/sgl-project/sglang/pull/22705
• @ColinZ22 made their first contribution in https://github.com/sgl-project/sglang/pull/22543
• @cyyc0310 made their first contribution in https://github.com/sgl-project/sglang/pull/22920
• @divyamagrawal06 made their first contribution in https://github.com/sgl-project/sglang/pull/23325
• @dyhsup made their first contribution in https://github.com/sgl-project/sglang/pull/22439
• @egvenediktov made their first contribution in https://github.com/sgl-project/sglang/pull/20520
• @erikwijmans made their first contribution in https://github.com/sgl-project/sglang/pull/21974
• @fengli1702 made their first contribution in https://github.com/sgl-project/sglang/pull/19143
• @fergusfinn made their first contribution in https://github.com/sgl-project/sglang/pull/21035
• @fortunecookiee made their first contribution in https://github.com/sgl-project/sglang/pull/20960
• @gxlvera made their first contribution in https://github.com/sgl-project/sglang/pull/19225
• @he-yufeng made their first contribution in https://github.com/sgl-project/sglang/pull/20739
• @Henson-Zh-Ali made their first contribution in https://github.com/sgl-project/sglang/pull/20522
• @icepoint666 made their first contribution in https://github.com/sgl-project/sglang/pull/22592
• @iridiumine made their first contribution in https://github.com/sgl-project/sglang/pull/20918
• @is-not made their first contribution in https://github.com/sgl-project/sglang/pull/18349
• @JasonHe-WQ made their first contribution in https://github.com/sgl-project/sglang/pull/21944
• @jh-nv made their first contribution in https://github.com/sgl-project/sglang/pull/21254
• @jiangyinzuo made their first contribution in https://github.com/sgl-project/sglang/pull/23169
• @JieTang66 made their first contribution in https://github.com/sgl-project/sglang/pull/23983
• @JoyFuture made their first contribution in https://github.com/sgl-project/sglang/pull/23808
• @jthakurH made their first contribution in https://github.com/sgl-project/sglang/pull/16793
• @kangyifei made their first contribution in https://github.com/sgl-project/sglang/pull/23241
• @kingkingleeljj made their first contribution in https://github.com/sgl-project/sglang/pull/20967
• @kkyyxhll made their first contribution in https://github.com/sgl-project/sglang/pull/23062
• @KrishnanPrash made their first contribution in https://github.com/sgl-project/sglang/pull/22175
• @lahmuller made their first contribution in https://github.com/sgl-project/sglang/pull/22625
• @lixuwei2333 made their first contribution in https://github.com/sgl-project/sglang/pull/22247
• @lkhl made their first contribution in https://github.com/sgl-project/sglang/pull/22431
• @loading66 made their first contribution in https://github.com/sgl-project/sglang/pull/22700
• @luccafong made their first contribution in https://github.com/sgl-project/sglang/pull/24165
• @mingyue300 made their first contribution in https://github.com/sgl-project/sglang/pull/21723
• @minosfuture made their first contribution in https://github.com/sgl-project/sglang/pull/23419
• @mispa-ms made their first contribution in https://github.com/sgl-project/sglang/pull/23097
• @mlleo made their first contribution in https://github.com/sgl-project/sglang/pull/23537
• @Napkin-AI made their first contribution in https://github.com/sgl-project/sglang/pull/23572
• @nvpohanh made their first contribution in https://github.com/sgl-project/sglang/pull/22852
• @officialasishkumar made their first contribution in https://github.com/sgl-project/sglang/pull/22600
• @opherlieber made their first contribution in https://github.com/sgl-project/sglang/pull/22547
• @ranjiewen made their first contribution in https://github.com/sgl-project/sglang/pull/21698
• @RichardoMrMu made their first contribution in https://github.com/sgl-project/sglang/pull/19545
• @robellliu-dev made their first contribution in https://github.com/sgl-project/sglang/pull/20835
• @SammLSH made their first contribution in https://github.com/sgl-project/sglang/pull/22089
• @Seven-Streams made their first contribution in https://github.com/sgl-project/sglang/pull/21722
• @shenxiul made their first contribution in https://github.com/sgl-project/sglang/pull/23327
• @siju-samuel made their first contribution in https://github.com/sgl-project/sglang/pull/23472
• @stepinto made their first contribution in https://github.com/sgl-project/sglang/pull/23478
• @tfhddd made their first contribution in https://github.com/sgl-project/sglang/pull/22029
• @vvagaytsev made their first contribution in https://github.com/sgl-project/sglang/pull/22363
• @WangHao-hw made their first contribution in https://github.com/sgl-project/sglang/pull/22778
• @Wen-xuan-Xu made their first contribution in https://github.com/sgl-project/sglang/pull/22923
• @xiaobochen-amd made their first contribution in https://github.com/sgl-project/sglang/pull/22626
• @yaya159456 made their first contribution in https://github.com/sgl-project/sglang/pull/21694
• @YMbmzy made their first contribution in https://github.com/sgl-project/sglang/pull/22049
• @yuki-brook made their first contribution in https://github.com/sgl-project/sglang/pull/18016
• @Zaire404 made their first contribution in https://github.com/sgl-project/sglang/pull/22982
• @ZeyuanChen2000 made their first contribution in https://github.com/sgl-project/sglang/pull/21543
• @zhaozx-cn made their first contribution in https://github.com/sgl-project/sglang/pull/22266
• @zhsurpass made their first contribution in https://github.com/sgl-project/sglang/pull/22697
• @zsj555 made their first contribution in https://github.com/sgl-project/sglang/pull/23454

Full Changelog: https://github.com/sgl-project/sglang/compare/v0.5.10.post1...v0.5.11

Full Changelog: https://github.com/sgl-project/sglang/compare/v0.5.10...v0.5.10.post1

Bumps flashinfer from v0.6.7.post2 to v0.6.7.post3 to resolve an issue in its jit cubin downloader.