# PEFT v0.20.0 - Product: PEFT (https://whatsnew.fyi/product/peft) - Vendor: Hugging Face - Date: 2026-07-28 - Version: v0.20.0 - Original notes: https://github.com/huggingface/peft/releases/tag/v0.20.0 - Permalink: https://whatsnew.fyi/product/peft/releases/v0.20.0 What's New is an index, not a publisher: every entry below links to the vendor's own release notes, which are the authoritative source. Entries are labelled where they are hand-curated sample data, pre-releases, or drawn from a secondary source such as a developer blog. Reuse: the summaries, labels and curation here are © What's New. Quote freely with attribution and a link back; wholesale republication of the corpus is not permitted — terms: https://whatsnew.fyi/terms. The vendors' own release notes remain their publishers'. --- - **added** — Add HiRA (Hadamard High-Rank Adaptation) method which multiplies low-rank product with frozen base weight using Hadamard product instead of additive combination - **added** — Add GLoRA (Generalized LoRA) method extending LoRA with configurable weight, activation, and bias adaptation - **added** — Add BEFT (Bias-Efficient Fine-Tuning) method that trains only bias terms, targeting the value projection by default for roughly 0.01% trainable parameters - **added** — Add MonteCLoRA method treating low-rank parameters as distributions and using Monte Carlo estimation for more robust training - **added** — Add VeLoRA method compressing intermediate activations with rank-1 sub-token projections to reduce activation memory - **added** — Add Uni-LoRA method using single global projection across the model for cross-layer parameter sharing - **added** — Add FRoD (Full-Rank Efficient Fine-Tuning with Rotational Degrees) method reconstructing weights with shared rotational subspaces and sparse trainable coefficients - **added** — Add MiCA method initializing from SVD of base weight and training only on smallest singular value directions - **added** — Add DEFT (Decompositional Efficient Fine-Tuning) method for text-to-image models splitting weight updates into projection and low-rank components - **added** — Add automatic LoRA target selection enhancement #### Highlights peft-v0 20 0 This release adds no less than nine new PEFT methods and puts a lot of work into the surrounding infrastructure, for example adding a new image generation benchmark for the method comparison suite and greatly improving the documentation structure. ##### New Methods ###### HiRA @hqsiswiliam added ["HiRA: Parameter-Efficient Hadamard High-Rank Adaptation for Large Language Models"](https://openreview.net/forum?id=TwJrTz9cRS) to PEFT (#2668). Instead of adding the low-rank product `BA` to the base weight, HiRA multiplies it elementwise (Hadamard product) with the frozen base weight. Because the base weight itself is full rank, the resulting update is no longer constrained to be low rank, while the trainable parameter count stays the same as LoRA's. ###### GLoRA @not-lain contributed GLoRA: ["One-for-All: Generalized LoRA for Parameter-Efficient Fine-Tuning"](https://arxiv.org/abs/2306.07967) in #3098. It is a flexible PEFT method that extends LoRA with configurable weight, activation, and bias adaptation, delivering richer fine-tuning with no extra inference cost. Use it when you need per-layer flexibility or stronger adaptation than vanilla LoRA. Skip it for non-Linear layers (e.g. Conv/Embedding) or when standard LoRA is already sufficient and simplicity matters. ###### BEFT @whubaichuan added ["BEFT: Bias-Efficient Fine-Tuning of Language Models"](https://arxiv.org/abs/2509.15974v2) in #3195. BEFT builds on the observation that fine-tuning bias terms alone can be competitive in low-data regimes, but goes further: rather than training *all* biases, it targets the value projection by default, as the authors found this to be most efficient. This brings the trainable parameter count down to roughly 0.01% of the total parameters. ###### MonteCLoRA @victor7246 integrated MonteCLoRA, ["Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation"](https://openreview.net/forum?id=2HFmicB8kh) in #2943. LoRA is known to be sensitive to hyperparameters like learning rate and batch size. This new LoRA variant addresses this by treating the low-rank parameters as a distribution rather than a point estimate, using Monte Carlo estimation to obtain a low-variance posterior estimate. This should make training noticeably more robust, at the cost of only _O(rank)_ additional parameters. ###### VeLoRA @roymiles added VeLoRA: ["Memory Efficient Training using Rank-1 Sub-Token Projections"](https://arxiv.org/abs/2405.17991) in #3159. Unlike most PEFT methods, this LoRA variant targets activation memory rather than parameter count: intermediate activations are split into sub-tokens and compressed with a fixed rank-1 projection before being cached for the backward pass, then reconstructed during backpropagation. Conceptually, it is similar to gradient checkpointing, with a lower memory saving but running faster. ###### Uni-LoRA @KaiyangLi1992 contributed Uni-LoRA: ["One Vector is All You Need"](https://arxiv.org/abs/2506.00799) in #3257. The paper shows that parameter-efficient LoRA variants like VeRA and VB-LoRA can all be described as projecting the full LoRA parameter space down to a much smaller subspace, differing only in the choice of projection. Uni-LoRA uses a single global projection across the whole model instead of layer-wise ones, which allows cross-layer parameter sharing and thus very low parameter counts. ###### FRoD @Bane-Elvin added FRoD, ["Full-Rank Efficient Fine-Tuning with Rotational Degrees for Fast Convergence"](https://arxiv.org/abs/2512.23485), in #3270. Instead of adding low-rank deltas like LoRA, it reconstructs selected weights with shared rotational subspaces and sparse trainable coefficients. It is especially useful when fast convergence and a higher full-rank capacity ceiling are im _[Truncated at 4000 characters — full notes: https://github.com/huggingface/peft/releases/tag/v0.20.0]_