PEFT v0.16.0

v0.16.0

0.16.0: LoRA-FA, RandLoRA, C³A, and much more

Added 8
  • Add LoRA-FA optimizer based on AdamW to increase memory efficiency of LoRA training
  • Add RandLoRA method that uses non-learnable random low rank matrices combined through learnable matrices, with support for quantized models with bitsandbytes
  • Add Circular Convolution Adaptation (C3A) as a new PEFT method to overcome low rank adaptation limits
  • Add support for LoRA with Conv2d layers where groups != 1 (requires rank r divisible by groups)
  • Add Intel Neural Compressor (INC) quantization support to LoRA
  • Add DoRA support for Conv1d layers
  • Add orthogonal weight initialization for LoRA via init_lora_weights='orthogonal'
  • Add Quantization-Aware LoRA training with GPTQ support
Changed 4
  • Refactor Orthogonal Finetuning (OFT) for improved speed and memory efficiency, but this is incompatible with old OFT checkpoints
  • Set required_grads_ of modules_to_save to True when used directly with inject_adapter for PEFT integrations
  • Update compatibility with Transformers vision language model refactor; apply prompt learning methods to vlm directly instead of vlm.language_model
  • Update prompt learning methods for Transformers attention mask refactoring, which may result in numerical differences for prefix tuning and affect models using 4d attention masks like Gemma
Fixed 1
  • Fix LoRA hotswapping with compiled models to no longer cause CUDA graph re-records

Highlights

peft-v0 16 0

New Methods
LoRA-FA

In #2468, @AaronZLT added the LoRA-FA optimizer to PEFT. This optimizer is based on AdamW and it increases memory efficiency of LoRA training. This means that you can train LoRA with less memory, or, with the same memory budget, use higher LoRA ranks, potentially getting better results.

RandLoRA

Thanks to @PaulAlbert31, a new PEFT method called RandLoRA was added to PEFT (#2464). Similarly to VeRA, it uses non-learnable random low rank matrices that are combined through learnable matrices. This way, RandLoRA can approximate full rank updates of the weights. Training models quantized with bitsandbytes is supported.

C³A

@Phoveran added Circular Convolution Adaptation, C3A, in #2577. This new PEFT method can overcome the limit of low rank adaptations as seen e.g. in LoRA while still promising to be fast and memory efficient.

Enhancements

Thanks to @gslama12 and @SP1029, LoRA now supports Conv2d layers with groups != 1. This requires the rank r being divisible by groups. See #2403 and #2567 for context.

@dsocek added support for Intel Neural Compressor (INC) quantization to LoRA in #2499.

DoRA now supports Conv1d layers thanks to @EskildAndersen (#2531).

Passing init_lora_weights="orthogonal" now enables orthogonal weight initialization for LoRA (#2498).

@gapsong brought us Quantization-Aware LoRA training in #2571. This can make QLoRA training more efficient, please check the included example. Right now, only GPTQ is supported.

There has been a big refactor of Orthogonal Finetuning, OFT, thanks to @zqiu24 (#2575). This makes the PEFT method run more quickly and require less memory. It is, however, incompatible with old OFT checkpoints. If you have old OFT checkpoints, either pin the PEFT version to <0.16.0 or retrain it with the new PEFT version.

Thanks to @keepdying, LoRA hotswapping with compiled models no longer leads to CUDA graph re-records (#2611).

Changes

Compatibility
  • #2481: The value of required_grads_ of modules_to_save is now set to True when used directly with inject_adapter. This is relevant for PEFT integrations, e.g. Transformers or Diffusers.
  • Due to a big refactor of vision language models (VLMs) in Transformers, the model architecture has been slightly adjusted. One consequence of this is that if you use a PEFT prompt learning method that is applied to vlm.language_model, it will no longer work, please apply it to vlm directly (see #2554 for context). Morever, the refactor results in different checkpoints. We managed to ensure backwards compatability in PEFT, i.e. old checkpoints can be loaded successfully. There is, however, no forward compatibility, i.e. loading checkpoints trained after the refactor is not possible with package versions from before the refactor. In this case, you need to upgrade PEFT and transformers. More context in #2574.
  • #2579: There have been bigger refactors in Transformers concerning attention masks. This required some changes on the PEFT side which can affect prompt learning methods. For prefix tuning specifically, this can result in numerical differences but overall performance should be the same. For other prompt learning methods, numerical values should be the same, except if the base model uses 4d attention masks, like Gemma. If you load old prompt learning checkpoints, please double-check that they still perform as expected, especially if they're trained on Gemma or similar models. If not, please re-train them or pin PEFT and transformers to previous versions (<0.16.0 and <4.52.0, respectively).
All Changes
New Contributors

Full Changelog: https://github.com/huggingface/peft/compare/v0.15.2...v0.16.0

View original

Upgraded? How did it go?

Discussion