Discussions

• 1 You must be logged in to vote

  🙏

  如何对数据集的训练集进行推理？

  huangkr03 asked May 12, 2025 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  请教下，一次评估多个模型，模型在数据集推理时，是串行的还是并行的，还是某种形式的并发态？

  Redias asked May 8, 2025 in Q&A · Unanswered

  

  

  0
• 4 You must be logged in to vote

  🙏

  Evaluating VLMs on text-only datasets.

  sbdzdz asked May 7, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  cache_size_limit reached when evaluating the accuracy

  WillLester asked Apr 28, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  在评估BoolQ时并不计算acc，只显示了extract_rate

  ZV-Liu asked Apr 10, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  AIME2024 dataset on opencompass is different from the dataset on huggingface

  ljliu started Mar 25, 2025 in View all discussions

  

  

  

  

  2
• 1 You must be logged in to vote

  🙌

  Deepseek-r1测试效果不好（Deepseek-r1's testing results weren't great）

  lissettecarlr started Feb 14, 2025 in Show and tell

  

  

  

  

  6
• 1 You must be logged in to vote

  🙏

  执行脚本没有反应

  rangehow asked Apr 8, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  如何在prompt template后加入某些字符？

  CurryxIaoHu asked Mar 25, 2025 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  使用Ollama框架测试AIME2024精度异常

  AaroAaron asked Mar 17, 2025 in Q&A · Unanswered

  

  

  3
• 1 You must be logged in to vote

  🙏

  使用VLLM推理速度慢，吞吐量低

  sunsn1997 asked Mar 11, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  请问hf-type字段只支持base和chat。那么请问Instruct模型应该选哪个？

  BigFishLi asked Mar 10, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  只评估&榜单结果评测方法

  suanflower asked Mar 3, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  使用配置文件启动后，如何断点继续？

  qing-yuan233 asked Feb 26, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  使用配置文件时，如何正确设置推理参数？

  qing-yuan233 asked Feb 25, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  使用配置文件运行时，如何指定并发任务量？

  qing-yuan233 asked Feb 24, 2025 in Q&A · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  不定数量round应该如何设置？

  xcfcode asked Jan 17, 2025 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  🙏

  使用LLaMA2-7b的基座模型进行mmlu评估，正确率有点低，求助

  ShenJiarun asked Jan 17, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  测试longbench报错，我该怎么在脚本中设置超过2048这个参数

  fxnie asked Jan 15, 2025 in Q&A · Unanswered

  

  

  2
• 1 You must be logged in to vote

  🙏

  使用opencompass调用vllm的模型，使用leval测评直接报错，怎么解决，求求了

  fxnie asked Jan 15, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  ppl和gen模式有什么区别，可以用通俗的语言解释一下吗

  xealml asked Nov 15, 2023 in Q&A · Answered

  

  

  

  

  7
• 1 You must be logged in to vote

  🙏

  一个数据集有多个名字有什么区别？

  1920993165 asked Dec 28, 2024 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  🙏

  跑MT-bench-101数据集时报错 No module named 'rouge'

  chaserRen asked Dec 25, 2024 in Q&A · Unanswered

  

  

  

  

  6
• 1 You must be logged in to vote

  🙏

  What is the use of first_capital_postprocess and last_capital_postprocess in text_postprocessors.py

  wangluyi asked Dec 26, 2024 in Q&A · Unanswered

  

  

  0
• 4 You must be logged in to vote

  🙏

  代码尝试使用 inspect.getargspec，但是在 Python 3.10中，inspect 模块已经没有了 getargspec 属性，因此导致了 AttributeError，请问如何解决？

  KogWin asked May 17, 2024 in Q&A · Unanswered

  

  

  1