Skip to content

Conversation

bmahabirbu
Copy link
Contributor

@bmahabirbu bmahabirbu commented Jun 10, 2025

What does this PR do?

see above

thanks to @axel7083 for his work so far in #3077
I think ill be able to use InferenceRuntimeSelect.svelte to help select the runtime and pass it to the playground creation IF there is no model services available to select. If so i think ill just point the user to create the service in the model services page

Screenshot / video of UI

so far these are pics of selecting the inference provider
Screenshot 2025-06-10 011247

What issues does this PR fix or reference?

fixes #2613

How to test this PR?

create multiple services with the same model and see if you can choose from it in the playground

@jeffmaury
Copy link
Collaborator

Got to test it but this is not what I had in mind: for me when selecting a model, you should first select the runtime type (all / llama-cpp /openvino) and based on your selection the list of available models is filtered. It seems to me aligned with what is described in the issue but asking @slemeur for confirmation

@bmahabirbu
Copy link
Contributor Author

Thanks @jeffmaury! Agreed I'll work on what you described in a separate pr.

Once that is done maybe we can revisit this idea later down the line once each model has multiple runtimes to pick from.

This maybe useful if a user has multiple services with different runtimes for a specific model they wanted to pick from.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implementation Selecting inferencing runtime when starting a playground
2 participants