Skip to content

Issues: open-compass/VLMEvalKit

Beta
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

103 Open 303 Closed
103 Open 303 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Unable to reproduce llava_v1.5_7b scores on several benchmarks including MME, TextVQA, POPE etc.
#1029 opened May 29, 2025 by sanghyunna
有什么方法可以从本地读取数据文件吗
#1028 opened May 29, 2025 by AdvancedBot
qwen2vl-7b推理报错
#1027 opened May 29, 2025 by AdvancedBot
2
InternVL2_5-26B not working
#1023 opened May 27, 2025 by JVal123
when using TextVQA_VAL，meet torch.OutOfMemoryError
#1021 opened May 27, 2025 by liuzhongling
2
Support for Heron-Bench Feature Request Extra attention is needed
#1011 opened May 25, 2025 by okojoalg
多卡推理qwen2.5-vl-7B的时候张量并行报错
#1009 opened May 23, 2025 by FontMLLM
2
How to evaluate MathVista with GPT-like closed-source api?
#1006 opened May 23, 2025 by TongkunGuan
1
单机多卡测试,shape对不上
#1002 opened May 21, 2025 by dulibubai
2
除了输入image的路径之外，能否将image的tensor作为model.generate()方法的输入？
#998 opened May 19, 2025 by fyqsama
1
Are there any problems with using vlmeval for the model evaluation of qwen2/2.5?
#997 opened May 19, 2025 by warmsnow-sh
1
Fine-tuned Model
#995 opened May 17, 2025 by JVal123
2
InternVL2_5-8B-BoN-8（VisualPRM）任务无法复现
#994 opened May 17, 2025 by YuanDaoze
1
support for think model
#989 opened May 12, 2025 by nutsintheshell
1
关于qwen2.5omni的性能
#988 opened May 12, 2025 by WenmuZhou
1
结果Excel能否保存每个样本正确或错误或者得分
#981 opened May 7, 2025 by bjzhb666
2
测试api based model时preprocess_message_with_role()会造成system prompt无限叠加 BUG Something isn't working
#977 opened May 4, 2025 by vvirgooo2
2
Adding LiveXiv - a live benchmark Feature Request Extra attention is needed
#976 opened May 3, 2025 by NimrodShabtay
1
Turn Off Saving Images in LMUData
#965 opened Apr 28, 2025 by moured
1
OCRBenchV2 Feature Request Extra attention is needed
#949 opened Apr 25, 2025 by JVal123
1
很多模型使用vlmevalkit评测不同的数据集时效果都有较大的差距
#941 opened Apr 23, 2025 by lanyuki
3
version of transfomers
#936 opened Apr 22, 2025 by The-Git-Rain
2
support for MV-Math Feature Request Extra attention is needed
#933 opened Apr 18, 2025 by SakurajimaMaiii
[bug] 反馈两个bug，模型和数据集用config.json配置会出现一些问题 WIP
#929 opened Apr 17, 2025 by gulixin0922
1
Error in analyzing video datasets with Qwen2.5-VL-3B-Instruct
#927 opened Apr 16, 2025 by DanialTaheri
ProTip! Exclude everything labeled bug with -label:bug.