finetune.md

Finetuning

We provide a simple finetuning commands (litgpt finetune_*) that instruction-finetune a pretrained model on datasets such as Alpaca, Dolly, and others. For more information on the supported instruction datasets and how to prepare your own custom datasets, please see the tutorials/prepare_dataset tutorials.

LitGPT currently supports the following finetuning methods:

litgpt finetune_full
litgpt finetune_lora
litgpt finetune_adapter
litgpt finetune_adapter_v2

 

Tip

To install all required dependencies before finetuning, first run pip install "litgpt[all]".  

The following section provides more details about these methods, including links for additional resources.

 

LitGPT finetuning commands

The section below provides additional information on the available and links to further resources.

 

Full finetuning

litgpt finetune_full

This method trains all model weight parameters and is the most memory-intensive finetuning technique in LitGPT.

More information and resources:

• the LitGPT tutorials/finetune_full tutorial

 

LoRA and QLoRA finetuning

litgpt finetune_lora stabilityai/stablelm-base-alpha-3b

LoRA and QLoRA are parameter-efficient finetuning technique that only require updating a small number of parameters, which makes this a more memory-efficienty alternative to full finetuning.

More information and resources:

• the LitGPT tutorials/finetune_lora tutorial
• the LoRA paper by (Hu et al. 2021)
• the conceptual tutorial Parameter-Efficient LLM Finetuning With Low-Rank Adaptation (LoRA)

 

Adapter finetuning

litgpt finetune_adapter stabilityai/stablelm-base-alpha-3b

or

litgpt finetune_adapter_v2 stabilityai/stablelm-base-alpha-3b

Similar to LoRA, adapter finetuning is a parameter-efficient finetuning technique that only requires training a small subset of weight parameters, making this finetuning method more memory-efficient than full-parameter finetuning.

More information and resources:

• the LitGPT tutorials/finetune_adapter tutorial
• the Llama-Adapter (Gao et al. 2023) and Llama-Adapter v2 (Zhang et al. 2023) papers that originally introduces these methods
• the conceptual tutorial Understanding Parameter-Efficient Finetuning of Large Language Models: From Prefix Tuning to LLaMA-Adapters