rpc : use GGML_LOG_* for logging #16129
Conversation
github-actions
bot
added
examples
ggml
Add --verbose to rpc-server and configure the ggml logger to print debug messages only if set. Add helper macro LOG_DBG() which does an early check of the verbose flag before calling GGML_LOG_DEBUG(). Make sure we log a debug message for every server function.
rgerganov
force-pushed
the
rpc-fixlog
branch
from
cc8fee7 to
eda3897
Compare
| GGML_BACKEND_API void ggml_backend_rpc_start_server(ggml_backend_t backend, const char * endpoint, | ||
| const char * cache_dir, | ||
| const char * cache_dir, bool verbose, | ||
| size_t free_mem, size_t total_mem); |
There was a problem hiding this comment.
I think it's better to keep this function as it is and read the verbosity level from an environment variable, for example GGML_RPC_DEBUG. The idea is to keep the API clean from verbosity/log related settings.
This way, in order to see the internal ggml-rpc debug logs, you can run:
GGML_RPC_DEBUG=1 rpc-server ... --verboseThere was a problem hiding this comment.
GGML_RPC_DEBUG=1 rpc-server ... --verbose
Why do we need both env variable and --verbose flag in this case? I am fine with using env variable but I don't see what --verbose is supposed to do in this case. Maybe set the environment variable?
There was a problem hiding this comment.
Atm --verbose is redundant for rpc-server because the rpc-server has no verbose logging.
The convention in the rest of the tools is that --verbose enables verbosity for the tool-level logic. And if you additionally need extra debug logs from within libllama or libggml you control these with env variables. But these extra debug logs still require --verbose to be passed to the tool.
2 participants
Use GGML_LOG_DEBUG() for debug messages and configure the logger to print them only when rpc-server is started with --verbose. Make sure we log a debug message for every server function.