Skip to content

feat: added ability to select runtime for playground #3171

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

bmahabirbu
Copy link
Contributor

What does this PR do?

adds the ability to select the runtime for playground

Screenshot / video of UI

runtime

What issues does this PR fix or reference?

fixes #2613

How to test this PR?

@bmahabirbu bmahabirbu requested review from benoitf, jeffmaury and a team as code owners June 12, 2025 05:45
@bmahabirbu bmahabirbu requested review from feloy and SoniaSandler June 12, 2025 05:45
@bmahabirbu bmahabirbu force-pushed the playground-runtime branch 2 times, most recently from bc087d7 to bdabfb0 Compare June 12, 2025 06:06
@feloy
Copy link
Contributor

feloy commented Jun 12, 2025

A few questions:

  • openvino is not compatible with mac/arm. Should we list it in such platform?
  • what is the meaning of the all inference runtime?

@bmahabirbu
Copy link
Contributor Author

@feloy good questions! That would be a good idea to remove openvino from mac for now

'ALL' in this case is a placeholder for every runtime provider besides a hardcoded exclusion.

Essentially I wanted a way to list all the runtime providers in the playground creation and this made the most sense to me at the moment. There's most definitely a cleaner approach!

Copy link
Contributor

@gastoner gastoner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are missing none option.
The facebook model has none runtime but the filter for it is not there
image

Copy link
Contributor

@axel7083 axel7083 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The runtime select should be seen as a filter, no hardcoded all, it should be clearable, if no value is selected, then all models should be listed.

No need to define what mean "all"

@bmahabirbu
Copy link
Contributor Author

bmahabirbu commented Jun 17, 2025

Thanks, @axel7083. That's a good way to handle it! In that case, should the runtime selection dropdown be defaulted to having no value so the user can see all the models available?

@bmahabirbu
Copy link
Contributor Author

bmahabirbu commented Jun 17, 2025

@gastoner good catch I assumed "none" in this case was simply to have no selection. I did have a playground with resnet setup but since the model is used for object detection, I didn't get the chat to work. Maybe we can keep it excluded?

@bmahabirbu bmahabirbu force-pushed the playground-runtime branch from bdabfb0 to 385f589 Compare June 17, 2025 00:32
@axel7083
Copy link
Contributor

should the runtime selection dropdown be defaulted to having no value so the user can see all the models available?

Yes! By default no filtering

@bmahabirbu
Copy link
Contributor Author

@axel7083 updated!

Copy link
Collaborator

@jeffmaury jeffmaury left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have tested it and got 2 issues:

  • in the selector the none value is presented but this value is useless because there are no inference servers for those kind of models: they are just there so that recipes can load them independently from an inference server
  • when the inference runtime from the settings is different from all then it should be the default value of the selector here

@bmahabirbu bmahabirbu force-pushed the playground-runtime branch 2 times, most recently from 59dad1b to c590339 Compare June 20, 2025 07:24
@bmahabirbu
Copy link
Contributor Author

@feloy I believe that Mac support for OpenVINO might be coming soon, maybe we can keep it as a selection? Also I was trying to import @podman-desktop/api to the frontend script playground create to use the env.ismac but it seems frontend scripts only have the ability to pull from pd ui-svelte. Is there an easier approach to having ismac usable for frontend?

@bmahabirbu bmahabirbu force-pushed the playground-runtime branch from c590339 to 802118c Compare June 20, 2025 07:44
@bmahabirbu bmahabirbu force-pushed the playground-runtime branch from 802118c to c0281d0 Compare June 20, 2025 08:24
Copy link
Contributor

@axel7083 axel7083 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, tested on Windows, works as expected 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implementation Selecting inferencing runtime when starting a playground
5 participants