中文羊驼大模型二期 v3.0

[ymcui]

本次更新推出长上下文版模型Chinese-LLaMA-2-7B-16K和Chinese-LLaMA-2-13B-16K。

🚀 Chinese-LLaMA-2-7B-16K、Chinese-LLaMA-2-13B-16K

• 通过位置插值方法，在增量数据上进行训练，使模型获得长文本处理能力
• 标准版模型支持4K上下文，而新推出的长上下文版模型支持16K，且均可通过NTK方法进一步扩展（无需训练）
• 在长文本评测LongBench上，16K版模型相比标准版均有显著的性能提升，效果参阅长上下文版模型评测
• 主要处理长文本时，建议使用16K版；日常使用请选择标准版模型

注意事项

• transformers 4.31.0以上版本可直接加载16K模型；llama.cpp下则需要在推理时添加--rope-scale 4参数
• 使用长上下文版模型之前，务必参阅推理与部署表格，查看相关方法是否支持
• 与其他LLaMA模型一样，目前的Chinese-LLaMA-2-16K不适合直接拿来用于对话
• Chinese-Alpaca-2-16K将在下一个版本推出，敬请关注

其他更新

• 添加了Alpaca-2-13B输出样例（Add Alpaca-2-13B examples #131）
• llama.cpp：删除了中的-eps选项以适配GGUF新版模型（llama.cpp: remove -eps term due to deprecation in new gguf format #162）
• llama.cpp：更新了新版k-quant模型PPL结果（llama.cpp: update PPL w.r.t. new k-quant optimization #172）
• 更新训练策略（Update Training Guidelines #150）
• 更新openai_api_server_vllm.py（Update openai_api_server_vllm.py #176）
• 修复一处typo（Update README_vllm.md #177） contributed by @touale
• 添加了CMMLU推理脚本，使用方法请参考CMMLU推理脚本

For English release note, please refer to Discussion.

---

This discussion was created from the release 中文羊驼大模型二期 v3.0.

[ymcui]

Release Note v3.0

This update introduces the long-context models Chinese-LLaMA-2-7B-16K and Chinese-LLaMA-2-13B-16K.

🚀 Chinese-LLaMA-2-7B-16K & Chinese-LLaMA-2-13B-16K

• Using the Positional Interpolation Method, trained on incremental data, the model has gained the ability to handle long texts.
• The standard model supports a 4K context, while the newly introduced long-context model supports 16K. Both can be further expanded using the NTK method without training.
• On the LongBench long-text evaluation, the 16K model has a significant performance improvement compared to the standard version. For results, refer to https://github.com/ymcui/Chinese-LLaMA-Alpaca-2/blob/main/README_EN.md#long-context-model-evaluation
• For primarily handling long texts, it is recommended to use the 16K version; for daily use, please choose the standard model.

Notes

• Transformers version 4.31.0 and above can directly load the 16K model; for llama.cpp, you need to add the --rope-scale 4 parameter during inference.
• Before using the long-context model, please refer to the Inference and Deployment table to check if the relevant method is supported.
• Like other LLaMA models, the current Chinese-LLaMA-2-16K is not suitable for direct use in dialogues.
• Chinese-Alpaca-2-16K will be introduced in the next version. Stay tuned.

Other Updates

• Added Alpaca-2-13B output sample (Add Alpaca-2-13B examples #131).
• llama.cpp: Removed the -eps option to adapt to the new GGUF model (llama.cpp: remove -eps term due to deprecation in new gguf format #162).
• llama.cpp: Updated the new k-quant model PPL results (llama.cpp: update PPL w.r.t. new k-quant optimization #172).
• Updated training strategy (Update Training Guidelines #150).
• Updated openai_api_server_vllm.py (Update openai_api_server_vllm.py #176).
• Fixed a typo (Update README_vllm.md #177) contributed by @touale.
• Added CMMLU inference script. For usage, please refer to CMMLU Inference Script.