Skip to content

Commit 72fc905

Browse files
ngxsonVJHack
authored andcommitted
server : add loading html page while model is loading (ggml-org#9468)
* Adding loading page for '/' server requests * set content when model is loading * removed loading html file * updated cmakelist * updated makefile * cleaned up whitespace * cleanup for PR removed error * updated server test to handle 503 HTML * updated server test to handle 503 HTML * ca†ch 503 before parsing json * revert test * account for both api and web browser requests * precommit corrections * eol fix * revert changes to pre-commit * removed print statement * made loading message more descriptive * also support .html files --------- Co-authored-by: VJHack <[email protected]> Co-authored-by: Vinesh Janarthanan <[email protected]>
1 parent 527951b commit 72fc905

File tree

4 files changed

+23
-2
lines changed

4 files changed

+23
-2
lines changed

Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1455,6 +1455,7 @@ llama-server: \
14551455
examples/server/system-prompts.js.hpp \
14561456
examples/server/prompt-formats.js.hpp \
14571457
examples/server/json-schema-to-grammar.mjs.hpp \
1458+
examples/server/loading.html.hpp \
14581459
common/json.hpp \
14591460
common/stb_image.h \
14601461
$(OBJ_ALL)

examples/server/CMakeLists.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ set(PUBLIC_ASSETS
3030
system-prompts.js
3131
prompt-formats.js
3232
json-schema-to-grammar.mjs
33+
loading.html
3334
)
3435

3536
foreach(asset ${PUBLIC_ASSETS})
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
<!DOCTYPE html>
2+
<html>
3+
<head>
4+
<meta http-equiv="refresh" content="5">
5+
</head>
6+
<body>
7+
<div id="loading">
8+
The model is loading. Please wait.<br/>
9+
The user interface will appear soon.
10+
</div>
11+
</body>
12+
</html>

examples/server/server.cpp

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@
2727
#include "system-prompts.js.hpp"
2828
#include "prompt-formats.js.hpp"
2929
#include "json-schema-to-grammar.mjs.hpp"
30+
#include "loading.html.hpp"
3031

3132
#include <atomic>
3233
#include <chrono>
@@ -2664,10 +2665,16 @@ int main(int argc, char ** argv) {
26642665
return false;
26652666
};
26662667

2667-
auto middleware_server_state = [&res_error, &state](const httplib::Request &, httplib::Response & res) {
2668+
auto middleware_server_state = [&res_error, &state](const httplib::Request & req, httplib::Response & res) {
26682669
server_state current_state = state.load();
26692670
if (current_state == SERVER_STATE_LOADING_MODEL) {
2670-
res_error(res, format_error_response("Loading model", ERROR_TYPE_UNAVAILABLE));
2671+
auto tmp = string_split(req.path, '.');
2672+
if (req.path == "/" || tmp.back() == "html") {
2673+
res.set_content(reinterpret_cast<const char*>(loading_html), loading_html_len, "text/html; charset=utf-8");
2674+
res.status = 503;
2675+
} else {
2676+
res_error(res, format_error_response("Loading model", ERROR_TYPE_UNAVAILABLE));
2677+
}
26712678
return false;
26722679
}
26732680
return true;

0 commit comments

Comments
 (0)