[ML] Default endpoint allocations are now configurable #127783
Conversation
Min and max allocations for the default endpoints' adaptive allocation settings are now configurable via a setting. This is intended to help new clusters running on laptops and in serverless. This does not automatically increase or reduce the default endpoints to those values - we still need a cluster reboot to set the values. Relate elastic#124653
prwhelan
added
>enhancement
:ml
|
Hi @prwhelan, I've created a changelog YAML for you. |
|
Pinging @elastic/ml-core (Team:ML) |
There was a problem hiding this comment.
If I understand correctly, these changes will allow modifying the default endpoints. I know we've brought up concerns about modifying the defaults because they affect everyone.
I think there's also consistency issues here because we persist the default inference endpoints to the inference index. We only do that if the id is missing. So if a user does a GET _inference/_all, we'll persist the default inference endpoints the first time, then if they update the settings, reboot, and then issue another GET _inference/_all we won't persist them again and I don't think they'll see the new values they set via the settings.
I'm not totally sure but I suspect it will also affect usage beyond just retrieving the default endpoints.
We'd need to delete the default endpoints and recreate them. I think we should revisit whether we need to persist the default endpoints at all but that's another discussion 😄
| 0, | ||
| 0, | ||
| 32, | ||
| Setting.Property.Dynamic, |
There was a problem hiding this comment.
I think dynamic will only be useful if we're going to allow changes without having to reboot. I think we typically add a addSettingsUpdateConsumer call so we can listen for the changes for example:
|
We discussed this and determined that what we are trying to accomplish - setting the default endpoint min allocations to 1 so that an ML node is always active - can be implemented using the existing hardware profiles by setting the min ML node to 1. We can revisit this as needed |
3 participants
Min and max allocations for the default endpoints' adaptive allocation settings are now configurable via a setting.
This is intended to help new clusters running on laptops and in serverless.
This does not automatically increase or reduce the default endpoints to those values - we still need a cluster reboot to set the values.
Relate #124653