# PEFT v0.17.0 — 0.17.0: SHiRA, MiSS, LoRA for MoE, and more

- Product: PEFT (https://whatsnew.fyi/product/peft)
- Vendor: Hugging Face
- Date: 2025-08-01
- Version: v0.17.0
- Original notes: https://github.com/huggingface/peft/releases/tag/v0.17.0
- Permalink: https://whatsnew.fyi/product/peft/releases/v0.17.0

What's New is an index, not a publisher: every entry below links to the vendor's own release notes, which are the authoritative source. Entries are labelled where they are hand-curated sample data, pre-releases, or drawn from a secondary source such as a developer blog.

Reuse: the summaries, labels and curation here are © What's New. Quote freely with attribution and a link back; wholesale republication of the corpus is not permitted — terms: https://whatsnew.fyi/terms. The vendors' own release notes remain their publishers'.

---

- **added** — Add Sparse High Rank Adapters (SHiRA) method for improved performance over LoRAs, especially when using multiple adapters
- **added** — Add MiSS (Matrix Shard Sharing) method as an evolution of Bone with improved performance and memory efficiency
- **added** — Enable LoRA to target nn.Parameter directly using target_parameters config attribute, useful for models with Mixture of Expert layers
- **added** — Support injecting adapters based on a state_dict without needing to specify target_modules in the config
- **fixed** — Fix bug in prompt learning methods where modules_to_save was ignored, preventing classification layers from being trained or stored
- **fixed** — Fix create mask function signature change in transformers 4.53.1
- **fixed** — Fix faulty OFT parameter device test
- **fixed** — Allow peft_type to be a string
- **fixed** — Fix prefix tuning after transformers PR 38635
- **deprecated** — Deprecate Bone method in favor of MiSS, to be removed in PEFT v0.19.0

#### Highlights

<img width="1248" height="560" alt="peft-v0 17 0" src="https://github.com/user-attachments/assets/a206c099-10ee-4c13-80c1-0de7ed1df5cf" />

##### New Methods

###### SHiRA

@kkb-code contributed [Sparse High Rank Adapters](https://huggingface.co/docs/peft/main/en/package_reference/shira) (SHiRA, [paper](https://huggingface.co/papers/2406.13175)) which promise to offer a potential gain in performance over LoRAs - especially the concept loss when using multiple adapters is improved. Since the adapters only train on 1-2% of the weights and are inherently sparse, switching between adapters may be cheaper than with LoRAs. (#2584)

###### MiSS

 @JL-er added a new PEFT method, MiSS ([Matrix Shard Sharing](https://arxiv.org/abs/2409.15371)) in #2604. This method is an evolution of [Bone](https://huggingface.co/docs/peft/package_reference/bone), which, according to our [PEFT method comparison benchmark](https://huggingface.co/spaces/peft-internal-testing/PEFT-method-comparison), gives excellent results when it comes to performance and memory efficiency. If you haven't tried it, you should do so now.

At the same time, Bone will be deprecated in favor of MiSS and will be removed in PEFT v0.19.0. If you already have a Bone checkpoint, you can use [`scripts/convert-bone-to-miss.py`](https://github.com/huggingface/peft/tree/main/scripts/convert-bone-to-miss.py) to convert it into a MiSS checkpoint and proceed with training using MiSS.

##### Enhancements

###### LoRA for `nn.Parameter`

LoRA is now able to target `nn.Parameter` directly (#2638, #2665)! Ever had this complicated `nn.Module` with promising parameters inside but it was too custom to be supported by your favorite fine-tuning library? No worries, now you can target `nn.Parameters` directly using the [`target_parameters`](https://huggingface.co/docs/peft/main/en/developer_guides/lora#targeting-nnparameter-directly) config attribute which works similarly to `target_modules`.

This option can be especially useful for models with **Mixture of Expert** (MoE) layers, as those often use `nn.Parameter`s directly and cannot be targeted with `target_modules`. For example, for the [Llama4 family of models](https://huggingface.co/collections/meta-llama/llama-4-67f0c30d9fe03840bc9d0164), use the following config to target the MoE weights:

```python
config = LoraConfig(
    ...,
    target_modules=[],  # <= prevent targeting any modules
    target_parameters=["feed_forward.experts.down_proj", "feed_forward.experts.gate_up_proj"],
)
```

Note that this feature is still experimental as it comes with a few caveats and therefore might change in the future. Also, MoE weights with many experts can be quite huge, so expect a higher memory usage than compared to targeting normal `nn.Linear` layers.

###### Injecting adapters based on a `state_dict`

Sometimes, it is possible that there is a PEFT adapter checkpoint but the corresponding PEFT config is not known for whatever reason. To inject the PEFT layers for this checkpoint, you would usually have to reverse-engineer the corresponding PEFT config, most notably the `target_modules` argument, based on the `state_dict` from the checkpoint. This can be cumbersome and error prone. To avoid this, it is also possible to call `inject_adapter_in_model` and pass the loaded `state_dict` as an argument:

```python
from safetensors.torch import load_file
from peft import LoraConfig, inject_adapter_in_model

model = ...
state_dict = load_file(<path-to-safetensors-file>)
lora_config = LoraConfig()  # <= no need to specify further
model = inject_adapter_in_model(lora_config, model, state_dict=state_dict)
```

Find more on [`state_dict` based injection in the docs](https://huggingface.co/docs/peft/main/en/developer_guides/low_level_api#injection-based-on-a-statedict).

##### Changes

###### Compatibility

A bug in prompt learning methods caused `modules_to_save` to be ignored. Especially clas

_[Truncated at 4000 characters — full notes: https://github.com/huggingface/peft/releases/tag/v0.17.0]_