[Badcase]: 函数调用出现不正常token（iNdEx）

[abiaoa1314]

Model Series

Qwen2.5

What are the models used?

Qwen2.5-14B-int4

What is the scenario where the problem happened?

Qwen2.5-14B-int4 in transformers

Is this badcase known and can it be solved using avaiable techniques?

• I have followed the GitHub README.
• I have checked the Qwen documentation and cannot find a solution there.
• I have checked the documentation of the related framework and cannot find useful information.
• I have searched the issues and there is not a similar one.

Information about environment

OS: Windows 10
Python: Python 3.11.9
GPUs: 1 * 4060Ti
NVIDIA driver: 555.99(from nvidia-smi)
CUDA compiler: 12.1 (from nvcc -V)
PyTorch: 2.2.1+cu121 (from python -c "import troch; print(torch.version)")

Description

出现iNdEx不正常的token，做function call的时候，太不稳定了。

[github-actions]

This issue has been automatically marked as inactive due to lack of recent activity. Should you believe it remains unresolved and warrants attention, kindly leave a comment on this thread.

[ChiNoel-osu]

我可以确认14B版本（原生精度）确有此问题。Tool Call不太稳定。
7B版本没有这个问题。使用的同样的生成参数多次测试得出的结论。

[jklj077]

what model is Qwen2.5-14B-int4?