Skip to content

Issues: open-compass/VLMEvalKit

Beta
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

100 Open 303 Closed
100 Open 303 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

InternVL2_5-26B not working
#1023 opened May 27, 2025 by JVal123
when using TextVQA_VAL，meet torch.OutOfMemoryError
#1021 opened May 27, 2025 by liuzhongling
1
Support for Heron-Bench Feature Request Extra attention is needed
#1011 opened May 25, 2025 by okojoalg
多卡推理qwen2.5-vl-7B的时候张量并行报错
#1009 opened May 23, 2025 by FontMLLM
1
How to evaluate MathVista with GPT-like closed-source api?
#1006 opened May 23, 2025 by TongkunGuan
1
单机多卡测试,shape对不上
#1002 opened May 21, 2025 by dulibubai
2
除了输入image的路径之外，能否将image的tensor作为model.generate()方法的输入？
#998 opened May 19, 2025 by fyqsama
1
Are there any problems with using vlmeval for the model evaluation of qwen2/2.5?
#997 opened May 19, 2025 by warmsnow-sh
1
Fine-tuned Model
#995 opened May 17, 2025 by JVal123
2
InternVL2_5-8B-BoN-8（VisualPRM）任务无法复现
#994 opened May 17, 2025 by YuanDaoze
1
support for think model
#989 opened May 12, 2025 by nutsintheshell
1
关于qwen2.5omni的性能
#988 opened May 12, 2025 by WenmuZhou
1
结果Excel能否保存每个样本正确或错误或者得分
#981 opened May 7, 2025 by bjzhb666
2
测试api based model时preprocess_message_with_role()会造成system prompt无限叠加 BUG Something isn't working
#977 opened May 4, 2025 by vvirgooo2
2
Adding LiveXiv - a live benchmark Feature Request Extra attention is needed
#976 opened May 3, 2025 by NimrodShabtay
1
Turn Off Saving Images in LMUData
#965 opened Apr 28, 2025 by moured
1
OCRBenchV2 Feature Request Extra attention is needed
#949 opened Apr 25, 2025 by JVal123
1
很多模型使用vlmevalkit评测不同的数据集时效果都有较大的差距
#941 opened Apr 23, 2025 by lanyuki
3
version of transfomers
#936 opened Apr 22, 2025 by The-Git-Rain
2
support for MV-Math Feature Request Extra attention is needed
#933 opened Apr 18, 2025 by SakurajimaMaiii
[bug] 反馈两个bug，模型和数据集用config.json配置会出现一些问题 WIP
#929 opened Apr 17, 2025 by gulixin0922
1
Error in analyzing video datasets with Qwen2.5-VL-3B-Instruct
#927 opened Apr 16, 2025 by DanialTaheri
Qwen2.5-VL-3B-Instruct在很多评测数据集的指标与官方榜单差距较大
#907 opened Apr 11, 2025 by LIRENDA621
3
通过LMDeployAPI方式部署Qwen2-VL-7B-Instruct，测精度
#902 opened Apr 9, 2025 by simplew2011
Model load error in Ola model
#900 opened Apr 9, 2025 by zhizhou57
ProTip! Updated in the last three days: updated:>2025-05-24.