Skip to content

Conversation

@ilopezluna
Copy link
Contributor

Adds ability to query models from remote registries via ?remote=true query parameter.

  • Added registry client to model manager for remote operations
  • Split model retrieval into local/remote handlers
  • Made Tags field optional in JSON responses

GET /models/{name}?remote=true - Fetch model from remote registry

@doringeman We initially discussed creating a common type for local and remote models, but I ultimately decided against it because I don’t see a strong need.

We use different clients to retrieve local vs. remote models (the distribution client for local and the registry client for remote), and each returns its own type, which makes sense given their responsibilities.

For example, in the case of local models, it's important to return a Model in order to provide GGUFPath()

@ilopezluna ilopezluna requested a review from a team June 10, 2025 10:59
Copy link

@p1-0tr p1-0tr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kiview kiview requested a review from Copilot June 10, 2025 11:04
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds support for fetching models from remote registries via a ?remote=true flag and makes the Tags field optional in responses.

  • Introduces a registry client alongside the existing distribution client.
  • Splits handleGetModel into getLocalModel and getRemoteModel paths.
  • Updates the Model JSON schema to omit tags when empty.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
pkg/inference/models/manager_test.go Adds TestHandleGetModel covering both local and remote retrieval but has import/alias mismatches.
pkg/inference/models/manager.go Registers a registryClient, parses remote query param, and delegates to getLocalModel/getRemoteModel.
pkg/inference/models/api.go Updates Model.Tags JSON tag to use omitempty.
Comments suppressed due to low confidence (4)

pkg/inference/models/manager_test.go:7

  • Test uses strings.Contains and filepath.Join but misses imports for "strings" and "path/filepath"; add these imports to the import block.
net/http/httptest

pkg/inference/models/manager.go:283

  • [nitpick] Remote model mapping initializes Tags as empty, dropping any tags present in the registry model; consider populating this slice from model.Tags() or equivalent source.
Tags:    make([]string, 0),

pkg/inference/models/manager_test.go:152

  • Imported registry package is aliased as reg, but code calls registry.New(). Change to reg.New() to match the import alias.
server := httptest.NewServer(registry.New())

pkg/inference/models/manager.go:204

  • Missing import for the strconv package required for ParseBool. Add import "strconv".
if val, err := strconv.ParseBool(r.URL.Query().Get("remote")); err != nil {

@ilopezluna ilopezluna requested a review from doringeman June 10, 2025 12:50
Copy link
Contributor

@doringeman doringeman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM!

@ilopezluna ilopezluna merged commit 9861625 into main Jun 10, 2025
4 checks passed
@ilopezluna ilopezluna deleted the get-remote-models branch June 10, 2025 13:04
ericcurtin referenced this pull request in ericcurtin/model-runner Sep 21, 2025
…ontent (#72)

fix: use empty string instead of nullptr for templates requiring non null
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants