Skip to content

fix: /model picker lags badly with many provider models #2077

Description

@keyarr

Problem

When a provider exposes many models (100+), opening /model and navigating the picker with the arrow keys becomes extremely laggy one keypress takes noticeably long to move focus.

Cause

Two independent O(n²) hot spots combined to make every keystroke expensive:

  1. getModelOptions() rebuilt the option list on every keystroke. While the picker is open, each focus change re-renders ModelPicker, and the effort/display resolution path calls isGenuineSwitchProfileValuegetModelOptions() on every render; even for ordinary model ids that can never be cross-profile switch entries.

  2. getModelOptions() itself was O(n²) for catalog-backed routes. optionMatchesModel ran getRouteCatalogModelOption (a full catalog scan) inside an options.some(...) loop, and getCatalogOptionValue re-filtered the entire catalog per entry. On a ~50+ entry static catalog using 9router, was practically impossible to select another model due to lag.

Fix

  • hasOptionValue hoists the catalog lookup out of the per-option loop.
  • Duplicate api-names are precomputed once into a Set (getDuplicateCatalogApiNames) instead of re-filtering per entry.
  • isGenuineSwitchProfileValue short-circuits for values that don't start with the __switch_profile__: prefix (semantics preserved, including the documented literal-prefixed-id edge case).

Measured impact: getModelOptions() on the 100+ entry NVIDIA NIM catalog went from 43.5ms → 2.0ms (~22x), and no option-list rebuild happens per keystroke anymore.

Steps to reproduce

  1. Configure a provider with a large model catalog (OpenAI-compatible endpoint with 100+ discovered models).
  2. Run the CLI, open /model.
  3. Hold or tap the down arrow — navigation lags badly before the fix.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions