merge docs #9657
Open
merge docs #9657
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
lugimzzz
reviewed
Dec 18, 2024
llm/docs/merge_model.md
Outdated
|
|
||
| ## 2.大模型融合算法介绍 | ||
| ### 2.1 Linear | ||
| [Linear](https://arxiv.org/abs/2203.05482):融合方法通过对多个微调模型的权重进行线性加权平均实现模型的合并。这种方法基于以下假设：微调后的模型通常位于低误差盆地，线性组合权重能够保持其性能和特性。Linear 方法简单高效。 |
llm/docs/merge_model.md
Outdated
| --output_path ../checkpoints/model_merge | ||
|
|
||
| ``` | ||
| <summary>  脚本参数介绍</summary><div> |
llm/docs/merge_model.md
Outdated
|
|
||
| - PaddlePaddle 3.0-beta | ||
| - PaddleNLP 3.0.0b2 | ||
| - PaddleSlim develop |
llm/docs/merge_model.md
Outdated
| - **缓解对齐代价**：在对预训练模型进行微调 sft 后得到微调模型，之后我们通常进行 RLHF。RLHF 被证明有效提升了 LLMs 的用户友好性，但它会引入一个对齐税（alignment tax），即在对齐人类偏好后模型的性能可能有所下降。模型融合可以缓解对齐税。 | ||
|
|
||
| ## 2.大模型融合算法介绍 | ||
| ### 2.1 Linear |
llm/docs/merge_model.md
Outdated
| ### 3.1 环境准备 | ||
|
|
||
| - PaddlePaddle 3.0-beta | ||
| - PaddleNLP 3.0.0b2 |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## develop #9657 +/- ##
========================================
Coverage 52.79% 52.79%
========================================
Files 718 718
Lines 112241 112241
========================================
Hits 59262 59262
Misses 52979 52979 ☔ View full report in Codecov by Sentry. |
lugimzzz
reviewed
Dec 20, 2024
llm/docs/mergekit.md
Outdated
| #### 2.2.1 环境准备 | ||
|
|
||
| - PaddlePaddle 3.0-beta | ||
| - PaddleNLP 3.0.0b2 |
llm/docs/mergekit.md
Outdated
| <summary>  模型参数（Model Parameters）</summary><div> | ||
|
|
||
| - `model_path_list`: 融合模型的路径或名称列表，需至少包含两个路径。默认为 `None`。 | ||
| - `model_path_str`: 融合模型名称或路径字符串,以逗号分隔，默认为 `None`。 |
There was a problem hiding this comment.
去掉model_path_str，这个主要是单测用的
llm/docs/mergekit.md
Outdated
| - **dare**: [DARE](https://arxiv.org/abs/2311.03099)论文中提出稀疏方式，根据设定概率，随机选择保留原始权重或设为0。可选超参 `reserve_p`、`rescale`。 | ||
| - **magprune**:[DELLA](https://arxiv.org/abs/2406.11617)论文中提出稀疏方式，根据权重绝对值大小给定不同保留概率，随机选择保留原始权重或设为0。可选超参 `reserve_p`、`rescale`、`epsilon`。 | ||
|
|
||
| ### 2.2 快速开始 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
PR changes
docs
Description
merge docs