-
Notifications
You must be signed in to change notification settings - Fork 40
FEATURE: llm quotas #1047
FEATURE: llm quotas #1047
Conversation
|
Are we able to provide guidelines for setting the max tokens? Some tooltips would be helpful here, i.e - for the non-technical admin,concepts such as tokens might be missed. They are probably left wondering "How many words does this actually mean?" or "How many times can I message the persona?" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM overall, just a few suggestions here and there
assets/javascripts/discourse/components/ai-llm-quota-editor.gjs
Outdated
Show resolved
Hide resolved
assets/javascripts/discourse/components/ai-quota-duration-selector.gjs
Outdated
Show resolved
Hide resolved
assets/javascripts/discourse/components/modal/ai-llm-quota-modal.gjs
Outdated
Show resolved
Hide resolved
assets/javascripts/discourse/components/modal/ai-llm-quota-modal.gjs
Outdated
Show resolved
Hide resolved
assets/javascripts/discourse/components/ai-quota-duration-selector.gjs
Outdated
Show resolved
Hide resolved
This introduces a new feature for per group quotas
we need to route through llm model for simplicity
Co-authored-by: Keegan George <[email protected]>
Co-authored-by: Keegan George <[email protected]>
Co-authored-by: Keegan George <[email protected]>
Co-authored-by: Keegan George <[email protected]>
Co-authored-by: Keegan George <[email protected]>
Co-authored-by: Keegan George <[email protected]>
Co-authored-by: Keegan George <[email protected]>
Co-authored-by: Keegan George <[email protected]>
Co-authored-by: Keegan George <[email protected]>
|
OK @keegangeorge I think it is all addressed! thanks for the review |
|
thanks @SaifMurtaza , will add more clarity, I am on the fence on implementing an optional "absolute quota" that is shared on the group... then you can guarantee you would never spend more than N$ on AI a day even if you have very large groups. |
|
As mentioned on dev, the use of |
keegangeorge
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Just two small fixes to fix linting and .discourse-compatibility needed before merging 🚀
| import { getOwner } from "@ember/owner"; | ||
| import { service } from "@ember/service"; | ||
| import I18n from "discourse-i18n"; | ||
| import { i18n } from "discourse-i18n"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See internal: /t/124366/17 we should pin discourse-ai for core < beta4 in .discourse-compatibility
assets/javascripts/discourse/components/ai-llm-quota-editor.gjs
Outdated
Show resolved
Hide resolved
Co-authored-by: Keegan George <[email protected]>
Co-authored-by: Keegan George <[email protected]>
Adds a comprehensive quota management system for LLM models that allows:
This system provides granular control over LLM API usage by allowing admins
to define limits on both total tokens and number of requests per group.
Supports multiple concurrent quotas per model and automatically handles
quota resets.