Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEATURE]MaxKB消耗的tokens数量和实际LLM消耗的tokens数量出入比较大 #488

Open
xiaobug0929 opened this issue May 21, 2024 · 4 comments
Open

[FEATURE]MaxKB消耗的tokens数量和实际LLM消耗的tokens数量出入比较大 #488

xiaobug0929 opened this issue May 21, 2024 · 4 comments
Assignees
@baixin513
Labels
类型:bug

Comments

@xiaobug0929
Copy link

MaxKB 版本

1.1.3

请描述您的需求或者改进建议

image
MaxKB消耗的tokens数量和实际LLM消耗的tokens数量出入比较大，应该是MaxKB内部相关的向量计算也参与了计算。建议额外提供一个返回值，专门提供实际LLM消耗的tokens数量

请描述你建议的实现方案

No response

附加信息

No response
@baixin513
Copy link
Contributor

感谢反馈，我们统一使用了 GPT-2的模型计算的 输入和输出的 tokens，并没有计算 embedding ， 与在线模型的API计算方式可能不一样。

@baixin513 baixin513 changed the title [FEATURE] [FEATURE]MaxKB消耗的tokens数量和实际LLM消耗的tokens数量出入比较大 May 21, 2024
@marxy
Copy link

marxy commented May 23, 2024 •
edited
Loading

image
为什么我用的v1.1.3版本没有显示token消耗量的？

@baixin513
Copy link
Contributor

v1.1.3

你用的什么模型 ？

@marxy
Copy link

marxy commented Jun 4, 2024

v1.1.3

你用的什么模型 ？

我用ollama v0.1.38部署的qwen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@baixin513 baixin513

Labels
类型:bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@marxy @xiaobug0929 @baixin513