Skip to content

model: Minicpmo #3023

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Mar 25, 2025
Merged

model: Minicpmo #3023

merged 2 commits into from
Mar 25, 2025

Conversation

mickqian
Copy link
Collaborator

@mickqian mickqian commented Jan 21, 2025 •
edited
Loading

Motivation

Support openbmb/MiniCPM-o-2_6

Modifications

Checklist

@mickqian mickqian marked this pull request as ready for review January 21, 2025 03:19
@mickqian mickqian marked this pull request as draft January 23, 2025 12:26
@mickqian mickqian force-pushed the minicpmo branch 4 times, most recently from 1764685 to de11d37 Compare January 29, 2025 07:58
@mickqian mickqian marked this pull request as ready for review January 29, 2025 08:56
mickqian
mickqian commented Jan 29, 2025
View reviewed changes
@yizhang2077 yizhang2077 self-assigned this Feb 7, 2025
@mickqian mickqian force-pushed the minicpmo branch 5 times, most recently from bc33318 to 78f5175 Compare February 26, 2025 02:11
@merrymercy merrymercy requested a review from HaiShaw as a code owner March 3, 2025 08:12
@zhaochenyang20 zhaochenyang20 mentioned this pull request Mar 4, 2025
Open
62 tasks
yizhang2077
yizhang2077 reviewed Mar 4, 2025
View reviewed changes
@yizhang2077
Copy link
Collaborator

I think we should add test .mp3 files to other places instead of adding to project directly.

@zhaochenyang20
Copy link
Collaborator

3000

some rename ( image -> multi modal data) is involved. Some of them can be done in another pr

no worry. we can do it here

@mickqian mickqian changed the title Minicpmo model: Minicpmo Mar 23, 2025
@mickqian mickqian force-pushed the minicpmo branch 4 times, most recently from 87f83d4 to d00dc4d Compare March 24, 2025 05:02
yizhang2077
yizhang2077 reviewed Mar 24, 2025
View reviewed changes
yizhang2077
yizhang2077 reviewed Mar 24, 2025
View reviewed changes
yizhang2077
yizhang2077 reviewed Mar 24, 2025
View reviewed changes
yizhang2077
yizhang2077 reviewed Mar 24, 2025
View reviewed changes
test/srt/test_vision_openai_server.py

return audio_response

def _test_audio_speech_completion(self):
Copy link
Collaborator

@yizhang2077 yizhang2077 Mar 24, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we directly move audio related test into TestMinicpmoServer?

@yizhang2077
Copy link
Collaborator

yizhang2077 commented Mar 24, 2025 •
edited
Loading

LGTM，after ci passes and unit test related issues have been solved, this PR can be merged. cc @zhaochenyang20 and plz paste MMMU benchmark result here~ @mickqian

@yizhang2077
Copy link
Collaborator

FYI, tts model in current Minicpmo is not used, as well as streaming audio embedding

@mickqian
Copy link
Collaborator Author

LGTM，after ci passes and unit test related issues have been solve, this PR can be merged. cc @zhaochenyang20 and plz paste MMMU benchmark result here~   @mickqian

accuracy
sglang 0.481
hf 0.49

@zhaochenyang20
Copy link
Collaborator

@yizhang2077 wait for yi's approval!

@zhaochenyang20
Copy link
Collaborator

great！let me rerun the CI and merge it asap

@zhyncs zhyncs merged commit 1e86457 into sgl-project:main Mar 25, 2025
1 of 18 checks passed
@yizhang2077
Copy link
Collaborator

@zhaochenyang20 we may need contributor to support audio benchmark

@zhaochenyang20
Copy link
Collaborator

great work1!

merrymercy
merrymercy reviewed Mar 28, 2025
View reviewed changes
Copy link
Contributor

@merrymercy merrymercy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move all model-specific files under python/sglang/srt/managers/multimodal_processors to python/sglang/srt/models/

@mickqian mickqian mentioned this pull request May 21, 2025
Open
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@merrymercy merrymercy merrymercy left review comments

@yizhang2077 yizhang2077 yizhang2077 approved these changes

@Ying1123 Ying1123 Awaiting requested review from Ying1123 Ying1123 is a code owner

@zhyncs zhyncs Awaiting requested review from zhyncs zhyncs is a code owner

@hnyls2002 hnyls2002 Awaiting requested review from hnyls2002 hnyls2002 is a code owner

@ispobock ispobock Awaiting requested review from ispobock ispobock is a code owner

@ByronHsu ByronHsu Awaiting requested review from ByronHsu ByronHsu is a code owner

@HandH1998 HandH1998 Awaiting requested review from HandH1998

@BBuf BBuf Awaiting requested review from BBuf

@HaiShaw HaiShaw Awaiting requested review from HaiShaw

@zhaochenyang20 zhaochenyang20 Awaiting requested review from zhaochenyang20 zhaochenyang20 is a code owner

@xiezhq-hermann xiezhq-hermann Awaiting requested review from xiezhq-hermann xiezhq-hermann is a code owner

Assignees

@yizhang2077 yizhang2077

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@mickqian @yizhang2077 @zhaochenyang20 @WangVertex @merrymercy @zhyncs