# Accelerate v1.8.0 — v1.8.0: FSDPv2 + FP8, Regional Compilation for DeepSpeed, Faster Distributed Training on Intel CPUs, ipex.optimize deprecation - Product: Accelerate (https://whatsnew.fyi/product/accelerate) - Vendor: Hugging Face - Date: 2025-06-19 - Version: v1.8.0 - Original notes: https://github.com/huggingface/accelerate/releases/tag/v1.8.0 - Permalink: https://whatsnew.fyi/product/accelerate/releases/v1.8.0 What's New is an index, not a publisher: every entry below links to the vendor's own release notes, which are the authoritative source. Entries are labelled where they are hand-curated sample data, pre-releases, or drawn from a secondary source such as a developer blog. Reuse: the summaries, labels and curation here are © What's New. Quote freely with attribution and a link back; wholesale republication of the corpus is not permitted — terms: https://whatsnew.fyi/terms. The vendors' own release notes remain their publishers'. --- - **changed** — Simplified FSDPv2 model preparation to reduce composition complexity and improve user experience - **added** — Added FP8 support for distributed training - **changed** — Updated CCL_WORKER_COUNT variable and added KMP parameters for Intel CPU users to improve distributed training performance - **added** — Added support for regional compilation with the DeepSpeed engine - **deprecated** — ipex.optimize is deprecated in favor of PyTorch optimizations - **changed** — Expanded and stabilized support for Intel XPUs including FSDP2, big model inference, and tensor parallelism - **added** — Added SwanLab as an experiment tracking backend - **changed** — Deferred all tracker initializations to prevent premature setup of distributed environments - **fixed** — Fixed bf16 training with tensor parallelism - **changed** — Improved FP8 handling with and without DeepSpeed - **changed** — Updated Gaudi runners - **removed** — Removed torch_ccl dependency - **added** — Added support for standalone mode when default port is occupied on single node - **added** — Added kwargs support to optimizer, scheduler, and dataloader in accelerator().load_state() function - **fixed** — Fixed list object has no attribute keys error - **fixed** — Fixed missing te.LayerNorm in intel_transformer_engine - **added** — Added fp8_e5m2 support in dtype_byte_size - **added** — Added DeepSpeed automatic gradient accumulation - **added** — Added device-agnostic GradScaler for XPU support - **added** — Added use of datasets.IterableDataset shard if possible #### FSDPv2 refactor + FP8 support We've simplified how to prepare FSDPv2 models, as there were too many ways to compose FSDP2 with other features (e.g., FP8, torch.compile, activation checkpointing, etc.). Although the setup is now more restrictive, it leads to fewer errors and a more performant user experience. We’ve also added support for FP8. You can read about the results [here](https://github.com/huggingface/accelerate/tree/main/examples/fsdp2). Thanks to @S1ro1 for this contribution! * [FSDP2] Refactor + FP8 by @S1ro1 in https://github.com/huggingface/accelerate/pull/3585 #### Faster Distributed Training on Intel CPUs We updated the `CCL_WORKER_COUNT` variable and added `KMP` parameters for Intel CPU users. This significantly improves distributed training performance (e.g., Tensor Parallelism), with up to a 40% speed-up on Intel 4th Gen Xeon when training transformer TP models. * Set ccl and KMP param in simple launch by @jiqing-feng in https://github.com/huggingface/accelerate/pull/3575 #### Regional Compilation for DeepSpeed We added support for regional compilation with the DeepSpeed engine. DeepSpeed’s .compile() modifies models in-place using torch.nn.Module.compile(...), rather than the out-of-place torch.compile(...), so we had to account for that. Thanks @IlyasMoutawwakil for this feature! * Fix deepspeed regional compilation by @IlyasMoutawwakil in https://github.com/huggingface/accelerate/pull/3609 #### ipex.optimize deprecation `ipex.optimize` is being deprecated. Most optimizations have been upstreamed to PyTorch, and future improvements will land there directly. For users without PyTorch 2.8, we’ll continue to rely on IPEX for now. * remove ipex.optimize in accelerate by @yao-matrix in https://github.com/huggingface/accelerate/pull/3608 #### Better XPU Support We've greatly expanded and stabilized support for Intel XPUs: * enable fsdp2 benchmark on XPU by @yao-matrix in https://github.com/huggingface/accelerate/pull/3590 * enable big_model_inference on xpu by @yao-matrix in https://github.com/huggingface/accelerate/pull/3595 * enable test_load_checkpoint_and_dispatch_with_broadcast cases on XPU by @yao-matrix in * enable test_cli & test_example cases on XPU by @yao-matrix in https://github.com/huggingface/accelerate/pull/3578 * enable torchao and pippy test cases on XPU by @yao-matrix in https://github.com/huggingface/accelerate/pull/3599 * enable regional_compilation benchmark on xpu by @yao-matrix in https://github.com/huggingface/accelerate/pull/3592 * fix xpu 8bit value loading by @jiqing-feng in https://github.com/huggingface/accelerate/pull/3623 * add device-agnostic GradScaler by @yao-matrix in https://github.com/huggingface/accelerate/pull/3588 * add xpu support in TorchTensorParallelPlugin by @yao-matrix in https://github.com/huggingface/accelerate/pull/3627 #### Trackers We've added support for [SwanLab](https://github.com/SwanHubX/SwanLab) as an experiment tracking backend. Huge thanks to @ShaohonChen for this contribution ! We also deferred all tracker initializations to prevent premature setup of distributed environments. * Integrate SwanLab for offline/online experiment tracking for Accelerate by @ShaohonChen in https://github.com/huggingface/accelerate/pull/3605 * Fix: Defer Tracker Initialization to Prevent Premature Distributed Setup by @yuanjua in https://github.com/huggingface/accelerate/pull/3581 ##### What's Changed * Fix bf16 training with TP by @SunMarc in https://github.com/huggingface/accelerate/pull/3610 * better handle FP8 with and without deepspeed by @IlyasMoutawwakil in https://github.com/huggingface/accelerate/pull/3611 * Update Gaudi Runners by @IlyasMoutawwakil in https://github.com/huggingface/accelerate/pull/3593 * goodbye torch_ccl by @yao-matrix in https://github.com/huggingface/accelerate/pull/3580 * Add support for standalone mode when default port is occupied on single node by @laitifranz in https:// _[Truncated at 4000 characters — full notes: https://github.com/huggingface/accelerate/releases/tag/v1.8.0]_