Keep model loaded when using api

[nephi-dev]

There's any way to keep a model loaded when using the api?

Example, on a first request the model "sdxl.safetensors" is loaded, but on the second request it got loaded again, slowing the api

[asagi4]

What kind of workflows are you sending to the API? In my experience the model stays loaded as long as the prompt you send in doesn't change in ways that require reloading.

Are you using the same node ID for the checkpoint loader between prompts?

[asagi4]

Do you send a client id to the server? I think that may affect caching. You send it with the request when you connect to the websocket server and when you queue a prompt. If you don't send any, the server will just generate a random one.

[nephi-dev]

I'm not sending any, where do I send it?

[nephi-dev]

another thing @asagi4 I'm not using websocket, it's just the bare api

[asagi4]

You send it when you queue the prompt. I have code like this

def queue_prompt(prompt, client_id):
    p = {"prompt": prompt, "client_id": client_id}
    data = json.dumps(p).encode('utf-8')
    req =  urllib.request.Request("https://{}/prompt".format(server_address), data=data)
    return json.loads(urllib.request.urlopen(req).read())

Just generate the client id and keep it constant and that might help.

[nephi-dev]

figured out my problem, the ids was always adding up, got it fixed now, thanks for the help