Skip to content

Conversation

@p1-0tr
Copy link
Contributor

@p1-0tr p1-0tr commented Mar 27, 2025

Add support for listening on a unix socket in the example server.

To keep the change small, and avoid extending the set of cli flags I went with a slightly hacky way of passing the socket name to the server. I.e. if --host ends with .sock then create and listen on a unix socket, else listen on TCP. LMK if this is acceptable, or if it would be better to avoid overloading --host and create an additional cli flag?

@ngxson
Copy link
Collaborator

ngxson commented Mar 27, 2025

I think it also worth to mention this in arg.cpp

    add_opt(common_arg(
        {"--host"}, "HOST",
        string_format("ip address to listen, or bind to an UNIX socket if the address ends with .sock (default: %s)", params.hostname.c_str()),
        [](common_params & params, const std::string & value) {
            params.hostname = value;
        }
    ).set_examples({LLAMA_EXAMPLE_SERVER}).set_env("LLAMA_ARG_HOST"));

@p1-0tr p1-0tr force-pushed the ps-server-af-unix branch from 4516f3e to ddce0fe Compare March 27, 2025 16:05
@p1-0tr
Copy link
Contributor Author

p1-0tr commented Mar 27, 2025

I think it also worth to mention this in arg.cpp

    add_opt(common_arg(
        {"--host"}, "HOST",
        string_format("ip address to listen, or bind to an UNIX socket if the address ends with .sock (default: %s)", params.hostname.c_str()),
        [](common_params & params, const std::string & value) {
            params.hostname = value;
        }
    ).set_examples({LLAMA_EXAMPLE_SERVER}).set_env("LLAMA_ARG_HOST"));

Thanks, that's a great point :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants