server: --offline mode

[ochafik]

This adds --offline flag (env: LLAMA_OFFLINE) to use locally cached manifests & models if available, or fail

@ngxson I know you told me you've incubated something similar (here?) but since I was stuck in a plane with ~zero internet all day yesterday, I ended up hacking this together. Feel free to merge later or ignore in favour of your approach.

[ngxson]

I think we don't need to put this code block inside a long if..else like this. Instead, there is a more simple way:

    if (file_exists && offline) {
        LOG_INF("%s: using cached file (offline mode): %s\n", __func__, path.c_str());
        return true; // skip verification/downloading
    }

    // keep the rest of the code as-is

    common_load_model_from_url_headers headers;
    bool head_request_ok = false;
    bool should_download = !file_exists; // by default, we should download if the file does not exist

    // get ETag to see if the remote file has changed
    {

[ngxson]

Or even more fail-safe:

    if (offline) {
        if (file_exists) {
            // if offline mode is enabled and the file exists, we can use it
            LOG_INF("%s: using cached file (offline mode): %s\n", __func__, path.c_str());
            return true; // skip verification/downloading
        } else {
            LOG_ERR("%s: required file is not available in cache (offline mode): %s\n", __func__, path.c_str());
            return false;
        }
    }

[ngxson]

This code branch will be trigger when file_exists == false && offline == true, so technically one of the condition inside if (offline) below now become dead code.

Instead, you can move the if (offline) to above of if (file_exists). So if file exist, we completely skip reading metadata

[ochafik]

ah ofc, rearranged to skip, thanks!

[ngxson]

maybe we don't need this, because you can do LLAMA_OFFLINE=1 (not sure if the env is forwarded to child process though)