Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

自定义评测集报错 #3757

Open
cjl09 opened this issue Apr 3, 2025 · 1 comment
Open

自定义评测集报错 #3757

cjl09 opened this issue Apr 3, 2025 · 1 comment

Comments

@cjl09
Copy link

cjl09 commented Apr 3, 2025

Describe the bug
What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程,最好有截图)
git clone https://github.com/modelscope/ms-swift.git
cd ms-swift
pip install -e '.[eval]'

nproc_per_node=8
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
NPROC_PER_NODE=$nproc_per_node
swift eval
--model /mymodel
--eval_backend Native
--infer_backend pt
--eval_dataset general_qa
--dataset_args '{"general_qa": {"local_path": "/data", "subset_list": ["data"]}}'

运行自定义数据集eval时报错:
Traceback (most recent call last):
File "/ms-swift/swift/cli/eval.py", line 2, in
from swift.llm import eval_main
File "/ms-swift/swift/utils/import_utils.py", line 90, in getattr
module = self._get_module(self._class_to_module[name])
File "/ms-swift/swift/utils/import_utils.py", line 99, in _get_module
return importlib.import_module('.' + module_name, self.name)
File "/opt/conda/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/ms-swift/swift/llm/eval/init.py", line 2, in
from .eval import SwiftEval, eval_main
File "ms-swift1/swift/llm/eval/eval.py", line 6, in
from evalscope.constants import EvalBackend, EvalType
ImportError: cannot import name 'EvalBackend' from 'evalscope.constants' (/opt/conda/lib/python3.10/site-packages/evalscope/constants.py)

Your hardware and system info
Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息,如CUDA版本,系统,GPU型号和torch版本等)

===== Python 版本 =====
Python 3.10.13
===== PyTorch 版本 =====
2.5.1+cu118

Additional context
Add any other context about the problem here(在这里补充其他信息)
Name: evalscope
Version: 0.6.0

@cjl09
Copy link
Author

cjl09 commented Apr 3, 2025

Python 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import evalscope.constants as c
dir(c)
['AnswerKeys', 'ArenaMode', 'ArenaWinner', 'DEFAULT_ROOT_CACHE_DIR', 'DumpMode', 'Enum', 'EvalConfigKeys', 'EvalStage', 'FnCompletionParser', 'MetricMembers', 'MetricsConstant', 'OutputsStructure', 'PositionBiasMitigation', 'ReviewKeys', 'builtins', 'cached', 'doc', 'file', 'loader', 'name', 'package', 'spec']

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant