Automatically detect available features for LocalAI

[mwinkens]

Describe the feature you'd like to request

Currently you need to deselect features like image generation if the endpoint in your LocalAI is not available, because otherwise users are able to try to generate images and get an error 404. This could be automatically detected and deselected.

Describe the solution you'd like

I would like to automatically detect available features and disable unavailable ones.

Describe alternatives you've considered

You could also add a second OpenAI key and use OpenAI for features, that are not available yet

[MB-Finski]

This is actually a really good idea to implement! It'd also make sense to not register the providers if they're not available to avoid any confusing errors down the road.

[mwinkens]

Also you could detect if you receive a 401 unauthroized, so your api key is invalid (or you didn't enter any). I got this one while configuring this app in the nextcloud logs

[kyteinsky]

hello,
since the response from the /models does not contain any information if a particular model is for image/text/audio generation, we can't reliably predict it and disable providers, especially for local/custom providers. However, if the admin knows that the underlying provider does not have any image generation abilities, the task type can be disabled in the admin settings -> AI section.

Also you could detect if you receive a 401 unauthroized, so your api key is invalid (or you didn't enter any). I got this one while configuring this app in the nextcloud logs

the error should not be shown in the admin settings when configuring the app clearly saying invalid api key. This was already good for openai but this is also shown for non-openai providers.

[mwinkens]

This ticket is really old, it might be obsolete by now

[kyteinsky]

obsolete how do you mean?
what I commented is the current state of things in light of the feature request, and local AI is still used widely.
but the feature request will be closed anyway since it is not actionable, waiting for the linked PR to close it automatically.

[mwinkens]

the error should not be shown in the admin settings when configuring the app clearly saying invalid api key.

When I wrote this ticket, this wasn't the case. You entered the key, got no feedback and then just noticed 401 errors in the logs. This is the obsolete part.

But this ticket is about automatically detecting what capabilities the localAI endpoint has. Idea is to make test requests and automatically disable unavailable endpoints, like for example image generation. Maybe a "Test" button behind these capabilities would be an option?

I don't get what your PR has to do with this issue. Thank you for improving the 401 error for an invalid API token, but this doesn't close this issue (maybe part of it)

[kyteinsky]

When I wrote this ticket, this wasn't the case. You entered the key, got no feedback and then just noticed 401 errors in the logs. This is the obsolete part.

thanks for the clarification.

But this ticket is about automatically detecting what capabilities the localAI endpoint has. Idea is to make test requests and automatically disable unavailable endpoints, like for example image generation

The automatic part can be a bit undesired since image generation can take some time on local small hardware, or even incur costs if you're using a hosted non-OpenAI provider. A "Test" beside each feature sounds like a good middle ground.

[kyteinsky]

implemented in #252

[mwinkens]

Nice 👍