# PEFT v0.19.0 - Product: PEFT (https://whatsnew.fyi/product/peft) - Vendor: Hugging Face - Date: 2026-04-14 - Version: v0.19.0 - Original notes: https://github.com/huggingface/peft/releases/tag/v0.19.0 - Permalink: https://whatsnew.fyi/product/peft/releases/v0.19.0 What's New is an index, not a publisher: every entry below links to the vendor's own release notes, which are the authoritative source. Entries are labelled where they are hand-curated sample data, pre-releases, or drawn from a secondary source such as a developer blog. Reuse: the summaries, labels and curation here are © What's New. Quote freely with attribution and a link back; wholesale republication of the corpus is not permitted — terms: https://whatsnew.fyi/terms. The vendors' own release notes remain their publishers'. --- - **added** — GraLoRA method that subdivides base weights into smaller blocks and applies LoRA to them for increased expressiveness and improved performance at higher ranks - **added** — BD-LoRA method that implements LoRA weights in block-diagonal form to reduce communication overhead when using tensor parallelism - **added** — Cartridges method to train a prefix that compresses long context to short size and save on tokens - **added** — PVeRA method as an extension of VeRA that adds probabilistic element by sampling from shared parameters - **added** — PSOFT method that improves efficiency of orthogonal fine-tuning by constraining adaptation to low-rank principal subspace - **added** — Lily method with sophisticated parameter sharing scheme where A parameters are shared blockwise and B parameters are chosen in data-dependent way through a router - **added** — PEANuT method that adds small neural networks called weight-aware neural tweakers to the base model - **added** — TinyLoRA method for training extremely small number of parameters much lower than LoRA rank 1 - **added** — AdaMSS method that segments base weights into smaller subspaces with dynamic parameter budget assignment to less important subspaces during training - **added** — Functions to convert checkpoints of many non-LoRA PEFT methods into LoRA checkpoints for compatibility with other packages #### Highlights This PEFT release contains no less than nine new PEFT methods, described below. It also contains numerous enhancements that should make PEFT more useful to many users. peft-v0 19 0 ##### New Methods ###### GraLoRA @yeonjoon-jung01 added ["GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning"](https://arxiv.org/abs/2505.20355) to PEFT (#2851). This method subdivides the base weight into smaller blocks and applies LoRA to those. This more granular adaptation promises to increase expressiveness and improve performance, especially at higher ranks (64+), closing the gap to full fine-tuning. ###### BD-LoRA @Conzel contributed BD-LoRA: ["Block-Diagonal LoRA for Eliminating Communication Overhead in Tensor Parallel LoRA Serving"](https://openreview.net/forum?id=1cjLvtFOmL) (#2895). With BD-LoRA, the LoRA weights are implemented in a block-diagonal way. This allows to reduce communication overhead when using tensor parallelism (TP) and thus faster serving. There is an experiment branch for BD-LoRA support in vLLM: vllm-project/vllm#28136. ###### Cartridges Thanks to @kashif, PEFT now also supports [Cartridges](https://arxiv.org/abs/2506.06266) (#2953). The main purpose of this method is to train a prefix to [compress a long context to a short size](https://hazyresearch.stanford.edu/blog/2025-06-08-cartridges) and thus save on tokens. On a low level, this is similar to [prefix tuning](https://huggingface.co/docs/peft/package_reference/prefix_tuning). The PR also added an [example recipe](https://github.com/huggingface/peft/tree/main/examples/cartridge_self_study) to quickly get started. ###### PVeRA ["PVeRA: Probabilistic Vector-Based Random Matrix Adaptation"](https://arxiv.org/abs/2512.07703) was added to PEFT by @leofillioux in #2952. It is an extension of [VeRA](https://huggingface.co/docs/peft/package_reference/vera), a PEFT method that uses weight sharing between layers to be especially parameter efficient. PVeRA builds on top of that by adding a probabilistic element, sampling from the shared parameters and promising better performance overall. ###### PSOFT @fei407 added PSOFT, ["Efficient Orthogonal Fine-Tuning with Principal Subspace Adaptation"](https://openreview.net/forum?id=FSHrinMArK), to PEFT in #3037. Orthogonal fine-tuning techniques like [OFT](https://huggingface.co/docs/peft/package_reference/oft) and [BOFT](https://huggingface.co/docs/peft/package_reference/boft) are good at preserving the structure and thus capabilities of the underlying base model. PSOFT improves efficiency of this technique by constraining the adaptation to low-rank principal subspace. ###### Lily @yibozhong added Lily: ["Low-Rank Interconnected Adaptation across Layers"](https://arxiv.org/abs/2407.09946) to PEFT in #2563. Lily is on the surface similar to LoRA but has a sophisticated parameter sharing scheme. The A parameters are shared blockwise (e.g. 4 consecutive q_proj layers share the same A). There is a pool of B parameters that is shared globally, the actual B's are chosen in a data-dependent way through a router. This allows Lily to use higher ranks than LoRA while maintaining a low trainable parameter count. ###### PEANuT In #3084, ["PEANuT: Parameter-Efficient Adaptation with Weight-aware Neural Tweakers"](https://arxiv.org/abs/2410.01870) was added to PEFT, again by @yibozhong. PEANuT adds a small, neural net (so called weight-aware neural tweakers) to the base model. Compared to LoRA, this increases expressivity for the same trainable parameter count or allows to greatly lower the parameter count without sacrificing expressivity. This comes at the expensive of a higher memory requirement for the same parameter count and decreased speed. ###### TinyLoRA We have another serial contributor in @kashif, who also contribu _[Truncated at 4000 characters — full notes: https://github.com/huggingface/peft/releases/tag/v0.19.0]_