Error in use MiniGPT4-Video inference

[zqs010908]

The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead.
Traceback (most recent call last):
  File "/home/iiau-vln/miniconda3/envs/MiniGPT4video/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/iiau-vln/miniconda3/envs/MiniGPT4video/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/iiau-vln/.vscode-server/extensions/ms-python.debugpy-2024.10.0/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/iiau-vln/.vscode-server/extensions/ms-python.debugpy-2024.10.0/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/iiau-vln/.vscode-server/extensions/ms-python.debugpy-2024.10.0/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/home/iiau-vln/.vscode-server/extensions/ms-python.debugpy-2024.10.0/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/iiau-vln/.vscode-server/extensions/ms-python.debugpy-2024.10.0/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/iiau-vln/.vscode-server/extensions/ms-python.debugpy-2024.10.0/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/media/iiau-vln/data2/ws_zqs/minigpt4zsy/MiniGPT4-video/minigpt4_video_inference.py", line 186, in <module>
    pred=run(video_path,instruction,model,vis_processor,gen_subtitles=add_subtitles)
  File "/media/iiau-vln/data2/ws_zqs/minigpt4zsy/MiniGPT4-video/minigpt4_video_inference.py", line 135, in run
    answers = model.generate(prepared_images, prompt, max_new_tokens=args.max_new_tokens, do_sample=True, lengths=[length],num_beams=1)
  File "/home/iiau-vln/miniconda3/envs/MiniGPT4video/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/media/iiau-vln/data2/ws_zqs/minigpt4zsy/MiniGPT4-video/minigpt4/models/mini_gpt4_llama_v2.py", line 553, in generate
    outputs = self.llama_model.generate(
  File "/home/iiau-vln/miniconda3/envs/MiniGPT4video/lib/python3.9/site-packages/peft/peft_model.py", line 1148, in generate
    outputs = self.base_model.generate(*args, **kwargs)
  File "/home/iiau-vln/miniconda3/envs/MiniGPT4video/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/iiau-vln/miniconda3/envs/MiniGPT4video/lib/python3.9/site-packages/transformers/generation/utils.py", line 2032, in generate
    result = self._sample(
  File "/home/iiau-vln/miniconda3/envs/MiniGPT4video/lib/python3.9/site-packages/transformers/generation/utils.py", line 2990, in _sample
    outputs = self(**model_inputs, return_dict=True)
  File "/home/iiau-vln/miniconda3/envs/MiniGPT4video/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/iiau-vln/miniconda3/envs/MiniGPT4video/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/iiau-vln/miniconda3/envs/MiniGPT4video/lib/python3.9/site-packages/accelerate/hooks.py", line 169, in new_forward
    output = module._old_forward(*args, **kwargs)
  File "/media/iiau-vln/data2/ws_zqs/minigpt4zsy/MiniGPT4-video/minigpt4/models/modeling_mistral.py", line 1160, in forward
    outputs = self.model(
  File "/home/iiau-vln/miniconda3/envs/MiniGPT4video/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/iiau-vln/miniconda3/envs/MiniGPT4video/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/iiau-vln/miniconda3/envs/MiniGPT4video/lib/python3.9/site-packages/accelerate/hooks.py", line 169, in new_forward
    output = module._old_forward(*args, **kwargs)
  File "/media/iiau-vln/data2/ws_zqs/minigpt4zsy/MiniGPT4-video/minigpt4/models/modeling_mistral.py", line 986, in forward
    position_ids = position_ids.view(-1, seq_length).long()
RuntimeError: shape '[-1, 0]' is invalid for input of size 3028

I check the value of input_ids in minigpt4/models/modeling_mistral.py line 956 ,it is tensor([], device='cuda:0', size=(1, 0), dtype=torch.int64),It seems that the error occurred because of this，but I don't know why input_ids is empty

[ChengjieSheng]

报这个错是因为transformers版本太新，使用transformers 4.37.2可以正常运行

pip install transformer==4.37.2

参考链接
#OpenGVLab/InternVL#405 (comment)