What’s New

transformers v5.11.0

v5.11.0

Release v5.11.0

Added
  • Add DiffusionGemma model with block-autoregressive text generation using multi-canvas sampling for improved inference speed
  • Add DeepSeek-V3.2 model with DeepSeek Sparse Attention (DSA) for improved training and inference efficiency in long-context scenarios
  • Add fine-grained fp8/fp4 Triton kernel support
  • Add documentation for tensor parallelism support with continuous batching
  • Add max_requests_per_batch parameter to continuous batching
  • Add Cosmos3 model
Changed
  • Extend KernelConfig API to support n-to-1 module fusion and parameter transformation
  • Update FalconMamba fast-path warning to recommend pip install kernels instead of mamba-ssm
  • Use torchvision's native LANCZOS interpolation instead of PIL fallback
  • Support non-weight param names in TorchAo safetensors loading
Fixed
  • Fix dtype mismatch in NemotronH/Zamba2 Mamba2 CUDA-kernel path for out_proj
  • Fix model parallel beam search bugs in Qwen2-VL, Qwen2.5-VL, and Qwen3-VL MoE model families
  • Fix the offsets in processing
  • Fix trailing comma bug in DataCollatorForLanguageModeling example
  • Fix missing Gemma4Processor._compute_audio_num_tokens
  • Fix InternVL models
  • Fix regression introduced by previous changes
  • Fix fbgemm_fp8 to keep the current device aligned with the input tensor
  • Fix video classification pipeline video processor
  • Fix Cohere2MoeModel device map attribute error in tests

Release v5.11.0

New Model additions
DiffusionGemma

DiffusionGemma is engineered to reduce the sequential bottlenecks of standard causal language models by employing an encoder-decoder architecture specifically optimized for inference speed. During inference, DiffusionGemma leverages multi-canvas sampling, where rather than generating one token at a time, the model iteratively denoises a full block of tokens using a diffusion sampler. This block-autoregressive approach facilitates text generation at higher speeds compared to traditional sequential generation methods.

Links: Documentation

  • GPU go brr (#46540) by @gante in #46540
DeepSeek-V3.2

DeepSeek-V3.2-Exp is an experimental model from DeepSeek-AI that introduces DeepSeek Sparse Attention (DSA), a trainable, fine-grained sparse attention mechanism designed to improve training and inference efficiency in long-context scenarios. Built on top of DeepSeek-V3.1-Terminus with a 685B-parameter Mixture-of-Experts backbone, it reduces the quadratic cost of attention over long sequences by attending only to a selected subset of past tokens while maintaining virtually identical benchmark performance. The work was extended in DeepSeek-V3.2 which pairs DSA with scalable reinforcement learning and achieves gold-medal level results on competition math and competitive programming benchmarks.

Links: Documentation | Paper

  • Add deepseek 3.2 exp (#41251) by @ArthurZucker in #41251
Kernels

The KernelConfig API was extended to support n-to-1 module fusion and parameter transformation, simplifying how custom kernels are integrated with Transformers modules. Additional fixes include resolving a dtype mismatch in the Mamba2 CUDA kernel path for NemotronH/Zamba2, adding fine-grained fp8/fp4 Triton kernel support, and correcting the FalconMamba fast-path warning to recommend pip install kernels instead of mamba-ssm.

  • Extended & simplified n-to-1 kernel fusion via KernelConfig (#46339) by @michaelbenayoun in [#46339]
  • Triton finegrained fp8/fp4 (#46407) by @IlyasMoutawwakil in [#46407]
  • Fix dtype mismatch in NemotronH/Zamba2 Mamba2 CUDA-kernel path (out_proj) (#46487) by @yuekaizhang in [#46487]
  • fix(falcon_mamba): recommend pip install kernels in fast-path warning (#46343) by @Anai-Guo in [#46343]
Parallelization

Fixed model parallel beam search bugs in the Qwen2-VL, Qwen2.5-VL, and Qwen3-VL MoE model families, and added documentation for tensor parallelism support with continuous batching.

  • [docs] tp for continuous batching (#46019) by @stevhliu in [#46019]
  • revisit history parallel beam search tests to avoid unnecessary fix (#46495) by @kaixuanliu in [#46495]
  • fix qwen series VL model's model parallel bug (#46316) by @kaixuanliu in [#46316]
Bugfixes and improvements
  • Fix the offsets in processing (#46525) by @zucchini-nlp in [#46525]
  • Fix buggy action sha pin (#46534) by @ydshieh in [#46534]
  • Fix trailing comma bug in DataCollatorForLanguageModeling example (#46527) by @JemmaUZH in [#46527]
  • Fix missing Gemma4Processor._compute_audio_num_tokens (#46416) by @csantosbh in [#46416]
  • Fix InternVL models (#46524) by @hmellor in [#46524]
  • fix(afmoe): reduce tokens in test_compile_static_cache to avoid flaky bfloat16 drift (#46521) by @ydshieh in [#46521]
  • [CB] Add a "max_requests_per_batch" parameter (#46434) by @remi-or in [#46434]
  • revamp cv docs and fix rf-detr (#46219) by @merveenoyan in [#46219]
  • Update hub metadata (#46379) by @zucchini-nlp in [#46379]
  • extend DeepseekV4FlashIntegrationTest to non-cuda device (#46517) by @sywangyi in [#46517]
  • [docs] deepgemm (#46361) by @stevhliu in [#46361]
  • [fix] regression introduced by #45534 (#46456) by @eustlb in [#46456]
  • Use torchvision's native LANCZOS interpolation instead of PIL fallback (#46496) by @NicolasHug in [#46496]
  • Add debugging info in pr-ci-caller.yml (#46505) by @ydshieh in [#46505]
  • Fix tests: 'Cohere2MoeModel' object has no attribute 'hf_device_map' (#46337) by @kaixuanliu in [#46337]
  • Bump the actions group across 1 directory with 19 updates (#46414) by @dependabot[bot] in [#46414]
  • Log some information in .github/workflows/pr-ci-post-dashboard-link.yml (#46499) by @ydshieh in [#46499]
  • feat(quantizers): support non-weight param names in TorchAo safetensors loading (#46325) by @agesf in [#46325]
  • docs: fix typo in make_list_of_images docstring (#46469) by @ramkumar27072006 in [#46469]
  • add XPU expectation for deepseek_ocr2 model tests (#46492) by @kaixuanliu in [#46492]
  • Fix sapiens2 tests: add XPU device expectations (#46488) by @kaixuanliu in [#46488]
  • Add vLLM smoke test to CI (#46383) by @hmellor in [#46383]
  • extend deepseek v4 test to xpu (#46366) by @sywangyi in [#46366]
  • Added cosmos3 model (#46146) by @MaciejBalaNV in [#46146]
  • fbgemm_fp8:Keep the current device aligned with the input tensor (#46403) by @kaixuanliu in [#46403]
  • [Modular] Add no_inherit_decorators and fixup wrong RoPE related inheritances (#46440) by @Bissmella in [#46440]
  • skip deepgemm test except cuda (#46090) by @jiqing-feng in [#46090]
  • Fix/video classification pipeline video processor (#46256) by @J3r3myPerera in [#46256]
  • ci: less flaky test_assisted_decoding_matches_greedy_search_1_same (#46445) by @ydshieh in [#46445]
  • Fix flip_back graph break (#46344) by @guarin in [#46344]
  • Add the other processors to auto-mappings (#46046) by @zucchini-nlp in [#46046]
  • fix: compatibility with torch<=2.7 (#46393) by @andylin-hao in [#46393]
  • fix: remove dynamic per-actor Slack ID lookup in ssh-runner workflow (#46327) by @ydshieh in [#46327]
  • [docs] Romanian translation of pipeline_tutorial.md, pipeline_gradio.md, pipeline_webserver.md and add_new_pipeline.md. (#46388) by @filipinescu in [#46388]
  • [docs] gemma4 typos (#46351) by @stevhliu in [#46351]
  • [docs] padding-free training (#46333) by @stevhliu in [#46333]
  • fix[vLLM x v5]: Default untied embeddings in AudioFlamingo3 and VibeVoice (#46400) by @harshaljanjani in [#46400]
  • Fix deepspeed docker (#46108) by @SunMarc in [#46108]
  • Fix conversion for clip models (#46406) by @zucchini-nlp in [#46406]
  • ci: mention code quality failure in CI dashboard comment (#46415) by @ydshieh in [#46415]
  • Fix noisy logging from image_processing module aliases issue - 46298 (#46350) by @skshmjn in [#46350]
  • Raise tqdm minimum to 4.60 to match tqdm.contrib.logging import (#46397) by @n0gu-furiosa in [#46397]
  • fix(gemma4_unified): conversion script and config bugs (#46398) by @douglas-reid in [#46398]
  • [docs] remove sparsity from compressed-tensors (#46387) by @stevhliu in [#46387]
  • [CB] Fix crashes when fork is not possible (#46251) by @remi-or in [#46251]
  • Improve CI dashboard comment: rename and deduplicate (#46412) by @ydshieh in [#46412]
  • Fix missing f-string prefixes in error messages (#46354) by @joaopedroassad in [#46354]
  • Add workflow to post CI Grafana dashboard link to PR (#46410) by @ydshieh in [#46410]
  • [docs] Romanian translation of fast_tokenizers.md, custom_tokenizers.md, tokenizer_summary.md, image_processors.md and video_processors.md. (#46356) by @filipinescu in [#46356]
  • Clean up new models after release (#46092) by @zucchini-nlp in [#46092]
Significant community contributions

The following contributors have made significant changes to the library over the last release:

  • @ArthurZucker
    • Add deepseek 3.2 exp (#41251)
  • @gante
    • GPU go brr (#46540)
  • @merveenoyan
    • revamp cv docs and fix rf-detr (#46219)
  • @sgerrard
    • Quantization for small models (#46449)
  • @MaciejBalaNV
    • Added cosmos3 model (#46146)
  • @J3r3myPerera
    • Fix/video classification pipeline video processor (#46256)
  • @filipinescu
    • [docs] Romanian translation of pipeline_tutorial.md, pipeline_gradio.md, pipeline_webserver.md and add_new_pipeline.md. (#46388)
    • [docs] Romanian translation of fast_tokenizers.md, custom_tokenizers.md, tokenizer_summary.md, image_processors.md and video_processors.md. (#46356)
View original