Skip to content

Issues: open-compass/opencompass

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

188 Open 332 Closed
188 Open 332 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Bug] pid_params.py dumps not successfully.
#1580 opened Sep 29, 2024 by tonysy
2 tasks done
@tonysy
[Bug] It seems run wrong number of prompts when max-num-workers > 1.
#1579 opened Sep 29, 2024 by zhulinJulia24
2 tasks done
@MaiziXiao
[Bug] debug不显示日志
#1578 opened Sep 29, 2024 by HaltonJiang
2 tasks done
@tonysy
[Bug] Wildbench主观分数不太一致
#1572 opened Sep 27, 2024 by mary-0830
2 tasks done
@tonysy @bittersweet1999
[Bug] mtbench101：batch_size设置
#1571 opened Sep 27, 2024 by zhang-junjian
2 tasks done
@tonysy @bittersweet1999
1
[Feature] Support MMMLU backlog
#1560 opened Sep 24, 2024 by tonysy
1 task
@liushz
1
[Bug] HuggingFacewithChatTemplate
#1557 opened Sep 24, 2024 by ZCzzzzzz
2 tasks done
2
[Bug] 试图添加mlc后端遇到问题，找不到cuda
#1551 opened Sep 23, 2024 by XJY990705
2 tasks done
@bittersweet1999
[Bug] SciCode miss with_background Bug Something isn't working
#1540 opened Sep 18, 2024 by tonysy
2 tasks done
@MaiziXiao
[Bug] mbpp_plus.jsonl 在哪里，下载的zip中没有这个东西
#1538 opened Sep 18, 2024 by kkwhale7
2 tasks done
@MaiziXiao
[Feature] About a subset of academic rankings
#1537 opened Sep 18, 2024 by mary-0830
1 task
@MaiziXiao
3
[Bug] PreTrainedTokenizerFast inference with unexpected parameter token_type_ids Bug Something isn't working
#1533 opened Sep 18, 2024 by kkwhale7
2 tasks done
@MaiziXiao
3
[Bug] MTBench数据集读取出现拼接bug
#1531 opened Sep 18, 2024 by zhang-junjian
2 tasks done
@MaiziXiao
2
使用Qwen2-72B-Base测试GPQA数据集时报错：NotImplementedError: OpenAI does not support ppl-based evaluation yet, try gen-based instead.
#1526 opened Sep 12, 2024 by 13416157913
2 tasks done
@acylam
2
[Bug] Cibench generate matplotlib has a wrong weights which should be [175, 2, 1, 156]
#1524 opened Sep 12, 2024 by imaginistLi
2 tasks done
@acylam
您好，请教一个问题，请问OpenCompass官网公开学术榜单中，测试的Qwen2模型中temperature、repetition_penalty、、top_p这些参数是设置多少的？模型部署的方式是采用什么方式部署的？
#1523 opened Sep 12, 2024 by 13416157913
2 tasks done
@acylam
3
[Feature] 数据集版本问题
#1509 opened Sep 9, 2024 by bjzhb666
1 task
@bittersweet1999
2
[Feature] How to config to support auto awq int4 llama3 model
#1506 opened Sep 9, 2024 by FlexLaughing
1 task
@bittersweet1999
Based on the openai service of the local model deployed by vllm, it is hoped that the sampling parameters top_p, etc. can be passed in when the model is initialized
#1505 opened Sep 9, 2024 by BrenchCC
1 task
@bittersweet1999
[Feature] 为什么调整batch_size之后，显存占用并没有提升呢
#1498 opened Sep 6, 2024 by belle9217
1 task
@tonysy
[Feature] 咱们oc啥时候能支持下lighteval的评测呀？
#1495 opened Sep 6, 2024 by caijie12138
1 task
@tonysy
2
[Bug] 单机多卡评测最后一张卡一直卡住
#1494 opened Sep 5, 2024 by bjzhb666
2 tasks done
4
[Feature] mtbench基线结果 backlog
#1492 opened Sep 5, 2024 by DietDietDiet
1 task
@acylam
1
[Feature] 添加mlc-llm后端支持 backlog
#1488 opened Sep 5, 2024 by XJY990705
1 task
@acylam @liushz
2
[Bug] 使用最新版本的glm4-9b-chat在Lawbench上推理会重复生成
#1477 opened Sep 2, 2024 by LKAMING97
2 tasks done
@bittersweet1999
17
ProTip! Mix and match filters to narrow down what you’re looking for.