Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Docs]:预测demo中加载了两次模型参数，不符合逻辑 #9482

Open
williamPENG1 opened this issue Nov 22, 2024 · 4 comments
Open

[Docs]:预测demo中加载了两次模型参数，不符合逻辑 #9482

williamPENG1 opened this issue Nov 22, 2024 · 4 comments
Assignees
@DesmonDay
Labels
documentation Improvements or additions to documentation

Comments

@williamPENG1
Copy link

软件环境

- paddlepaddle:
- paddlepaddle-gpu: 
- paddlenlp:

详细描述

这个文档里，predict时加载了两次模型参数，第一次是原始模型，第二次是训练后的参数，按理说，只需要加载训练后的参数即可，是不是可以再完善一下
@williamPENG1 williamPENG1 added the documentation Improvements or additions to documentation label Nov 22, 2024
@williamPENG1 williamPENG1 changed the title [Docs]: PaddleNLP/blob/develop/slm/examples/text_matching/ernie_matching/predict_pointwise.py [Docs]:预测demo中加载了两次模型参数，不符合逻辑 Nov 22, 2024
@williamPENG1
Copy link
Author

文件位置： PaddleNLP/blob/develop/slm/examples/text_matching/ernie_matching/predict_pointwise.py

@williamPENG1
Copy link
Author

3a2cbea8d22833f77fcf76240856d994

@DrownFish19
Copy link
Collaborator

您好，这里两次加载的模型不是同一个模型，一个是预训练的模型，一个是matching的模型。

@williamPENG1
Copy link
Author

您好，这里两次加载的模型不是同一个模型，一个是预训练的模型，一个是matching的模型。

可是这样不符合生产场景推理逻辑，预训练模型就几十GB的显存。我理解是不是要提供一个更优雅的方式，更好的推理？最好把query和doc的模型dump逻辑也补充进来

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DesmonDay DesmonDay

Labels
documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@DrownFish19 @williamPENG1 @DesmonDay