Skip to content

Issues: CLUEbenchmark/SuperCLUE

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

35 Open 13 Closed
35 Open 13 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

断更了？？？
#48 opened Nov 9, 2024 by yiliangfang
1
请问评测的原理是什么呀，是人工打分的吗，如果是客观题是直接比较返回的答案的字符串，主观题是人工评判答案吗
#47 opened Jul 26, 2024 by starplatinum3
2
工具的评测是什么含义？ 是Function calling么，如果没有请添加此能力的评测。
#46 opened Apr 11, 2024 by goqw
1
Claude3有评测吗？
#45 opened Mar 28, 2024 by Pancat007
请问如何对自己做的大模型使用该指标进行测评呢？
#44 opened Mar 11, 2024 by AWangji
4
请问一月榜单呢
#42 opened Feb 3, 2024 by kindle939393
2
咨询一下，从测评报告来看，SuperCLUE是采用自动化方式的客观评估，是否可提供针对某一模型的可实际运行的自动化评测的python样例代码(api调用或者web)？
#40 opened Jan 4, 2024 by Romanzhang2024
Where to download the benchmark dataset?
#38 opened Nov 30, 2023 by zhimin-z
How to calculate the metrics from the table in the paper to the leaderboard?
#37 opened Nov 30, 2023 by zhimin-z
1
大模型升级方式
#36 opened Nov 27, 2023 by lukeup
Where to locate the SuperCLUE-LYB leaderboards?
#34 opened Nov 18, 2023 by zhimin-z
能否增加翻译的评估排名
#33 opened Oct 12, 2023 by lx0126z
任务规划和工具使用的评价标准是什么样的？
#32 opened Sep 26, 2023 by heibaidaolx123
1
c-eval是真的离谱，希望superclue能更新的稍微快一点，比如1-2周更新一次
#31 opened Sep 8, 2023 by iammeizu
2
anthropic拼错了
#30 opened Aug 8, 2023 by JerryJiang12923
求教一下 逻辑与推理 具体指哪方面？ 比如 "郭德纲2岁会看报,xxxx" ，请问郭德纲3岁会看书吗？ 这个属于推理还是语义理解能力？？
#29 opened Aug 7, 2023 by ArtificialZeng
请问可以把vicuna-33B模型加入评测吗？
#28 opened Aug 3, 2023 by Mr-wang2016
测评时如何与标准答案进行匹配
#27 opened Jul 2, 2023 by Starry-Hu
数据集开源吗？可以在哪里下载呢
#26 opened Jun 26, 2023 by vanshaw2017
3
关于prompt设计的问题
#25 opened Jun 24, 2023 by lrs1353281004
1
排名变化的原因是什么？
#24 opened Jun 20, 2023 by zhaojiawen-coding
1
test the 智源大模型吧
#23 opened Jun 13, 2023 by forkyguo
3
这里"idea-jiangzhiya"应该是"idea-jiangziya"吧？
#21 opened Jun 1, 2023 by ilongshan
1
Clarify which "Claude" is benchmarked?
#19 opened May 22, 2023 by jekbradbury
1
可以在superclue上测试自己的模型吗？
#18 opened May 19, 2023 by guozhiyao
2
ProTip! Exclude everything labeled bug with -label:bug.