Skip to content

Bug: Wrong model used for summarization despite config settings #110

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alozowski opened this issue May 23, 2025 · 0 comments
Open

Bug: Wrong model used for summarization despite config settings #110

alozowski opened this issue May 23, 2025 · 0 comments

Comments

@alozowski
Copy link
Collaborator

Description:
I encountered a bug where the model defined for summarization in the model_roles configuration is not being used. Instead, the model defined above is used, likely due to how step names are handled internally.

Configuration example:

model_roles:
  ingestion:
    - Qwen/Qwen2.5-VL-72B-Instruct
  summarization:
    - Qwen/Qwen2.5-72B-Instruct

Expected Behavior:
The summaries should be generated by the Qwen/Qwen2.5-72B-Instruct model, as specified under the summarization key.

Observed Behavior:
Instead, the summaries are being generated by Qwen/Qwen2.5-VL-72B-Instruct.

Investigation Notes:
This seems to be caused by the use of step_name="summarization_chunk" in summarization.py. It appears this step name is not matched with the "summarization" key in the config, and since no explicit model role is defined for "summarization_chunk", the first model listed (i.e., the ingestion model) is used by default. This behavior is consistent with the fallback mechanism in inference_engine.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant