Skip to content

Add/Enable vllm inference benchmark run on persistent TPUVM #625

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 49 commits into from
Apr 10, 2025
Merged

Add/Enable vllm inference benchmark run on persistent TPUVM #625

merged 49 commits into from
Apr 10, 2025

Conversation

ManfeiBai
Copy link
Collaborator

@ManfeiBai ManfeiBai commented Mar 19, 2025 •
edited
Loading

Description

Add/Enable vllm inference benchmark run on persistent TPUVM

Tests

one-shot test: http://shortn/_RgPDz42A4N or http://shortn/_g3d7a0xpbm

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run one-shot tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed.

@ManfeiBai ManfeiBai requested a review from tengyifei as a code owner March 20, 2025 18:12
tengyifei
tengyifei requested changes Mar 20, 2025
View reviewed changes
Copy link
Collaborator

@tengyifei tengyifei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update the PR description?

@ManfeiBai ManfeiBai changed the title Enable vllm inference benchmark run on persistant TPUVM Enable vllm inference benchmark run on persistent TPUVM Mar 27, 2025
@ManfeiBai
Copy link
Collaborator Author

Can you update the PR description?

Thanks, updated

tengyifei
tengyifei reviewed Mar 31, 2025
View reviewed changes
@ManfeiBai ManfeiBai marked this pull request as ready for review April 10, 2025 18:37
@richardsliu richardsliu requested review from richardsliu and removed request for richardsliu April 10, 2025 20:30
tengyifei
tengyifei approved these changes Apr 10, 2025
View reviewed changes
@ManfeiBai ManfeiBai changed the title Enable vllm inference benchmark run on persistent TPUVM Add/Enable vllm inference benchmark run on persistent TPUVM Apr 10, 2025
@ManfeiBai ManfeiBai merged commit 5d36d4e into master Apr 10, 2025
16 checks passed
@ManfeiBai ManfeiBai deleted the ManfeiBai-patch-11 branch April 10, 2025 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@richardsliu richardsliu richardsliu left review comments

@tengyifei tengyifei tengyifei approved these changes

@vanbasten23 vanbasten23 Awaiting requested review from vanbasten23

@zpcore zpcore Awaiting requested review from zpcore

@yixinshi yixinshi Awaiting requested review from yixinshi

@RissyRan RissyRan Awaiting requested review from RissyRan

@hyeygit hyeygit Awaiting requested review from hyeygit

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ManfeiBai @tengyifei @richardsliu