How to change 'modules_to_save' setting when reloading a lora finetuned model

[dengchengxifrank]

System Info

• transformers version: 4.36.2
• Platform: Linux-3.10.0-1160.49.1.el7.x86_64-x86_64-with-glibc2.17
• Python version: 3.9.19
• Huggingface_hub version: 0.24.6
• Safetensors version: 0.4.5
• Accelerate version: 0.21.0
• Accelerate config: not found
• PyTorch version (GPU?): 2.0.1+cu117 (True)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using GPU in script?:
• Using distributed or parallel set-up in script?:

Who can help?

@BenjaminBossan

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder
• My own task or dataset (give details below)

Reproduction

@BenjaminBossan 1. I use lora to finetune whisper,and get the model A. The settings are

config = LoraConfig(r=8, lora_alpha=16,target_modules=target_modules,modules_to_save=modules_to_save,lora_dropout=0.05, bias="none")
model = get_peft_model(model, config)

and then I change the source code of model A, I add an additional layer. I now want to train a model with an extra layer based on the lora trained model A. I use:

model_lora_path = "../lora_path/" + 'checkpoint-56416'

model = PeftModel.from_pretrained(model,model_lora_path,ignore_mismatched_sizes=True).cuda()

But the model LoraConfig's "modules_to_save" can not be changed, I want to store the additional layer in to 'adapter_model.safetensors' How can I change my code?
In short, I want to add parameters to modules_to_save in LoraConfig during the reload process based on the trained lora model so that the additional layer can be stored.

I tried to use model.peft_config['default'].modules_to_save.extend(modules_to_save) to add the “modules_to_save” but it doesn't work.

Expected behavior

Change reload lora model's LoraConfig settings

[BenjaminBossan]

What you'd need to do in this case is to modify the modules_to_save argument before loading the model. Doing it after the model was loaded is too late. I see 2 options here:

1. You can directly edit the adapter_config.json in your checkpoint directory.
2. You can load the config first using PeftConfig.from_pretrained(<checkpoint-path>). Then, you pass that config like so: model = PeftModel.from_pretrained(..., config=peft_config). This is the cleaner solution.

LMK if this works.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.