# Accelerate v1.13.0 — v1.13.0: Neuron support, IPEX removal, and distributed training fixes - Product: Accelerate (https://whatsnew.fyi/product/accelerate) - Vendor: Hugging Face - Date: 2026-03-04 - Version: v1.13.0 - Original notes: https://github.com/huggingface/accelerate/releases/tag/v1.13.0 - Permalink: https://whatsnew.fyi/product/accelerate/releases/v1.13.0 What's New is an index, not a publisher: every entry below links to the vendor's own release notes, which are the authoritative source. Entries are labelled where they are hand-curated sample data, pre-releases, or drawn from a secondary source such as a developer blog. Reuse: the summaries, labels and curation here are © What's New. Quote freely with attribution and a link back; wholesale republication of the corpus is not permitted — terms: https://whatsnew.fyi/terms. The vendors' own release notes remain their publishers'. --- - **added** — Support for AWS Neuron (Trainium/Inferentia) devices - **changed** — Use spawn instead of fork for XPU device - **removed** — Remove IPEX dependency - **changed** — Enhance new codes to XPU and make them device agnostic - **fixed** — Fix KMP_AFFINITY incorrectly set for non-CPU training - **fixed** — Upcast FSDP2 parameters only if requires_grad - **fixed** — Fix FSDP2 tied embedding errors with targeted ValueError guidance - **fixed** — Fix FSDP2 cannot load optimizer state using dcp - **fixed** — Fix crash in optimizer.step when fsdp2 is enabled and model is bfloat16 - **fixed** — Fix FSDP2 crash with ignored_params on torch < 2.7.0 - **fixed** — Fix loss computation example for DeepSpeed Sequence Parallelism - **fixed** — Error out if both CP and SP enabled in DeepSpeed - **changed** — DeepSpeed has its own process group - **fixed** — Skip device mesh creation when deepspeed and sp_size > 1 - **added** — Enable evaluation during deepspeed Sequence Parallel - **fixed** — Fix FP8 torchao default config with padding and FSDP2 all-gather support - **fixed** — Fix execution with Transformer Engine for FP8 - **deprecated** — Add MS-AMP deprecation warnings - **changed** — Accelerate now imports faster by deferring heavy dependencies - **changed** — Torch.compile hooks are disabled lazily ##### AWS Neuron support We now have support for AWS Neuron (Trainium/Inferentia) devices. Thanks @michaelbenayoun for adding this. - Neuron integration by @michaelbenayoun in https://github.com/huggingface/accelerate/pull/3935 ###### XPU Improvements We've removed IPEX dependency and improved device-agnostic code for XPU. - using spawn instead of fork for XPU device by @kaixuanliu in https://github.com/huggingface/accelerate/pull/3884 - Remove ipex by @yao-matrix in https://github.com/huggingface/accelerate/pull/3883 - enhance new codes to XPU, and make them be device agnostic by @yao-matrix in https://github.com/huggingface/accelerate/pull/3890 - Fix KMP_AFFINITY incorrectly set for non-CPU training by @hexfaker in https://github.com/huggingface/accelerate/pull/3912 ##### FSDP2 Improvements We've added a bunch of important fixes for FSDP2 users: upcasting only grad-requiring params, better tied embedding errors, DCP optimizer loading, bf16 optimizer step crash fix, and torch < 2.7.0 compatibility. - Upcast FSDP2 parameters only if requires_grad by @ojh31 in https://github.com/huggingface/accelerate/pull/3848 - Fix FSDP2 tied embedding errors with targeted ValueError guidance by @amanzoni1 in https://github.com/huggingface/accelerate/pull/3878 - bug: fsdp cannot load optimizer state using dcp by @flymin in https://github.com/huggingface/accelerate/pull/3904 - fix crash in optimizer.step when fsdp2 is enabled and model is bfloat16 by @sywangyi in https://github.com/huggingface/accelerate/pull/3905 - Fix FSDP2 crash with ignored_params on torch < 2.7.0 by @Mr-Neutr0n in https://github.com/huggingface/accelerate/pull/3924 ##### DeepSpeed Sequence Parallelism We've added several fixes to the DeepSpeed + Sequence Parallelism integration introduced in v1.12.0, including evaluation support during SP training and proper process group handling. - [SP] fix loss computation example by @kashif in https://github.com/huggingface/accelerate/pull/3858 - [SP and CP] error out if both CP and SP enabled by @kashif in https://github.com/huggingface/accelerate/pull/3862 - DeepSpeed has its own process group by @kashif in https://github.com/huggingface/accelerate/pull/3916 - [Deepspeed] skip device mesh creation when deepspeed and sp_size >1 by @kashif in https://github.com/huggingface/accelerate/pull/3914 - Enable evaluation during deepspeed Sequence Parallel by @jp1924 in https://github.com/huggingface/accelerate/pull/3917 ###### FP8 We've enhanced FP8 training. Thanks @shimizust for fixing torchao support. - Fix FP8 torchao default config with padding and FSDP2 all-gather support by @shimizust in https://github.com/huggingface/accelerate/pull/3831 - Fix execution with Transformer Engine by @ksivaman in https://github.com/huggingface/accelerate/pull/3852 - add MS-AMP deprecation warnings by @neha222222 in https://github.com/huggingface/accelerate/pull/3857 ###### Performance Accelerate now imports faster by deferring heavy dependencies, and torch.compile hooks are disabled lazily. - Faster import by @SunMarc in https://github.com/huggingface/accelerate/pull/3953 - lazy compile disable by @SunMarc in https://github.com/huggingface/accelerate/pull/3947 - Disable hook compile by @SunMarc in https://github.com/huggingface/accelerate/pull/3888 ### Minor fixes - Allow non-Tensor values in a batch with dispatch_batches=True by @tomaarsen in https://github.com/huggingface/accelerate/pull/3850 - fix module and optimizer parameter mismatch before prepare_tp_ by @naomili0924 in https://github.com/huggingface/accelerate/pull/3845 - Fix KeyError in extract_model_from_parallel for partial torch.compile by @amanzoni1 in https://github.com/huggingface/accelerate/pull/3881 - Fix hf_device_map device index comparison in prepare_model by @rezaqorbani in https://github.com/huggingface/accelerate/pull/3895 - Fix StatefulDataLoader KeyError with num_workers > 0 by @veeceey in https://github.com/huggingfa _[Truncated at 4000 characters — full notes: https://github.com/huggingface/accelerate/releases/tag/v1.13.0]_