Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add common models and common params unit test for llm predictor. #9711

Open
wants to merge 7 commits into
base: develop
Choose a base branch
from

Conversation

aooxin
Copy link

@aooxin aooxin commented Dec 27, 2024

PR types

Others

PR changes

Others

Description

添加llm推理中常见模型和常见参数的单测
在A30单卡暂时耗时如下:
15 passed in 1507.02s (0:18:27)

覆盖下面的模型

  模型名称
1 Qwen2.5-7B-Instruct-tiny-nhl2
2 Qwen1.5-MoE-A2.7B-Chat-tiny-nhl2
3 Llama-2-7b-chat-tiny-nhl2
4 Meta-Llama-3.1-8B-Instruct-tiny-nhl2

Qwen2.5-1.5B-Instruct在参数测试中覆盖
70B和72B的在多卡中测试

覆盖下面的参数

两个 model:Qwen2.5-1.5B-Instruct和meta-llama/Llama-3.2-3B-Instruct

参数名称
use_fake_parameter True
inference_model False
append_attn True

覆盖分布式多卡推理场景

  模型名称
2 Qwen2.5-72B-Instruct-tiny-nhl2
3 Llama-2-70b-chat-tiny-nhl2
4 Meta-Llama-3.1-70B-Instruct-tiny-nhl2

Copy link

paddle-bot bot commented Dec 27, 2024

Thanks for your contribution!

Copy link

codecov bot commented Dec 27, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 52.77%. Comparing base (3aa9f4c) to head (863ff0b).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #9711      +/-   ##
===========================================
- Coverage    53.30%   52.77%   -0.53%     
===========================================
  Files          724      718       -6     
  Lines       118271   112449    -5822     
===========================================
- Hits         63039    59350    -3689     
+ Misses       55232    53099    -2133     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yuanlehome yuanlehome self-requested a review January 2, 2025 02:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants