Skip to content

Improve the API host config mechanism for the convenience API #67

@ncoghlan

Description

@ncoghlan

A few folks have reported that they find the current API for configuring the global default client unintuitive. The current API is to pass the host connection details when retrieving the default client instance:

import lmstudio as lms
lms.get_default_client("some_api_host:4321")

This will either create the default client instance with the specified server details, or raise an exception indicating that the default client already exists, so the given details couldn't be applied.

One potential path for improvement would leave this part of the interface alone, and instead add api_host=... parameters to all of the convenience API functions. Given that change, the most common mechanism for setting the API host details would likely be:

import lmstudio as lms
model = lms.llm(api_host="some_api_host:4321")

The downside of that approach is that it's still far from clear that the parameter is configuring the default global client (and will fail if it already exists).

The main alternative path for improvement would just be to add an explicit configuration API:

import lmstudio as lms
lms.configure_default_client("some_api_host:4321")

As with the existing API, this would fail if the default client already existed. Unlike the existing API, it wouldn't create the default client itself, it would just specify the settings to use when it is eventually created by some other usage of the convenience API.

I'm personally leaning towards the second option, but I'm curious if anyone would like to argue for the "common convenience API parameter" approach.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions