Add callback mechanism for common logging

[matiaslin]

I would like to propose to add a logging callback mechanism similar to llama and ggml in common. This will not only further unify the logging system, but it will also allow the user to control llama, ggml and common logs using a callback.

As of master, if the user sets the callback via llama_log_set, the common logs will be agnostic to this callback. For example, if the user were to specify a callback via llama_log_set that prevents any logs from being printed out, there would still be some logs from common being outputted to stderr.

For the example above, the user can redirect stderr to /dev/null to parse out the remaining logs, but this is not ideal since this approach masks real errors to the user. Therefore, I believe it would be beneficial to have a logging callback mechanism in common.

I'm happy to do the work on the implementation, and I'm open to thoughts, comments and/or suggestions?

Thank you.

[ggerganov]

I don't see a use case for this - separating the common logs from the llama logs is the intended behavior.

You can disable common logs with -lv -1.