Is it possible to bind SGLang Scheduler to 0.0.0.0 or 127.0.0.1 for CRIU C/R Purpose #18295
Replies: 1 comment
-
|
Actually, I have found the configuration variable that I am looking for to bind these processes to loopback address instead of eth0. I will consider this discussion as resolved. Thank you. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Problem Description
My current concern is that it seems that SGLang does not have any way to adjust scheduler IP address bind to first network interface on container or machine.
When I run SGLang, only the SGLang inference endpoint is configurable by
--hostflag as shown below.I would like to set SGLang scheduler to bind on same address configured on
--hostor atleast bind to127.0.0.1. I am currently not sure if the open ports by scheduler is required for multi-node/pod setup.Is it possible to do this or any way to configure it?
My use-case is basically like this.
Right now, I am failing to restore process and forced to re-use the original address during checkpointing. It is not ideal since the idea is to allow SGLang to scale-out using the same set of checkpoint.
Thank you very much!
Beta Was this translation helpful? Give feedback.
All reactions