Skip to content

Conversation

@daniel-lxs
Copy link
Member

@daniel-lxs daniel-lxs commented Aug 18, 2025

Summary

This PR fixes the issue where LM Studio models appeared twice in the Provider Configuration Profile when they were both downloaded and loaded. This implementation addresses the feedback from @mechanicmuthu about LM Studio's JIT Model Loading feature.

Problem

The previous implementation fetched models from two different LM Studio APIs:

  • listDownloadedModels() - showed all models on disk
  • listLoaded() - showed models currently in memory

This caused duplicates when a model was both downloaded and loaded, with inconsistent keys (model.path vs model.modelKey).

Solution

This PR implements a case-insensitive deduplication approach that:

  • Keeps both API calls to support LM Studio's JIT Model Loading (available since v0.3.5)
  • Deduplicates models by checking if any existing model key contains the loaded model's identifier (case-insensitive)
  • Prefers loaded model data when duplicates are found, as it provides more accurate runtime information (contextLength)
  • Maintains compatibility with JIT loading where models can be automatically loaded on-demand

Benefits

  • ✅ Eliminates duplicates with smart deduplication
  • ✅ Supports JIT Model Loading feature (models can be auto-loaded when requested)
  • ✅ Shows all available models (both downloaded and loaded)
  • ✅ Provides accurate runtime metadata when models are loaded
  • ✅ Handles auto-unload after 60-minute timeout gracefully

Testing

  • All existing tests pass
  • Added new test case for deduplication logic
  • Verified that duplicates are properly handled

Context

This replaces PR #7141 which only showed loaded models and didn't align with LM Studio's JIT loading capabilities.

Fixes #6954


Important

Implements case-insensitive deduplication in getLMStudioModels() to prevent duplicate model entries, preferring loaded models for accurate runtime data.

  • Behavior:
    • Implements case-insensitive deduplication in getLMStudioModels() in lmstudio.ts to prevent duplicate model entries.
    • Prefers loaded model data over downloaded when duplicates are found.
    • Ensures compatibility with JIT Model Loading by keeping both API calls.
  • Testing:
    • Adds test cases in lmstudio.test.ts for deduplication logic, including path-based matching and distinct name handling.
    • Verifies that duplicates are properly handled and runtime metadata is accurate.
  • Misc:
    • Handles auto-unload after 60-minute timeout gracefully.

This description was created by Ellipsis for 6aba8e5. You can customize this summary. It will automatically update as commits are pushed.

…cation

- Keep both listDownloadedModels and listLoaded APIs to support JIT loading
- Implement case-insensitive deduplication to prevent duplicates
- When duplicates are found, prefer loaded model data for accurate runtime info
- Add test coverage for deduplication logic
- Addresses feedback about LM Studio's JIT Model Loading feature (v0.3.5+)

Fixes #6954
@daniel-lxs daniel-lxs requested review from cte, jr and mrubens as code owners August 18, 2025 14:40
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. bug Something isn't working labels Aug 18, 2025
Copy link
Contributor

@roomote roomote bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution! I've reviewed the changes and found some issues that need attention. While the approach to support JIT Model Loading is good, the deduplication logic has some critical flaws that could cause incorrect model matching.

expect(result).toEqual({ [mockRawModel.modelKey]: expectedParsedModel })
})

it("should deduplicate models when both downloaded and loaded", async () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good test case! However, could we add more edge cases to ensure robustness? For example:

  • Models with similar names ("llama-3" vs "llama-3.1")
  • Multiple loaded models with various duplicate scenarios
  • Case sensitivity edge cases
  • Models with special characters in their paths/keys

- When a loaded model ID is found in any downloaded model key (case-insensitive)
- Remove the downloaded model and replace with the loaded model
- This ensures loaded models with runtime info take precedence
- Updated tests to verify the correct deduplication behavior
@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Aug 18, 2025
- Enhanced deduplication to use path segment matching instead of simple substring
- Prevents false positives like 'llama' matching 'codellama'
- Added comprehensive test cases for edge cases and multiple scenarios
- Maintains support for JIT Model Loading feature
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Aug 18, 2025
@daniel-lxs
Copy link
Member Author

Hey @mechanicmuthu, what do you think of this approach?

@daniel-lxs daniel-lxs moved this from Triage to PR [Needs Review] in Roo Code Roadmap Aug 18, 2025
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Aug 18, 2025
@mrubens mrubens merged commit e7e827a into main Aug 18, 2025
16 checks passed
@mrubens mrubens deleted the fix/lmstudio-deduplication branch August 18, 2025 15:17
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Aug 18, 2025
@github-project-automation github-project-automation bot moved this from PR [Needs Review] to Done in Roo Code Roadmap Aug 18, 2025
@hannesrudolph hannesrudolph added PR - Needs Review and removed Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. labels Aug 18, 2025
@mechanicmuthu
Copy link

mechanicmuthu commented Aug 18, 2025

Hey @daniel-lxs

Hey @mechanicmuthu, what do you think of this approach?

Autoloading of an unloaded model is working and tested now by me.

However, the duplicate issues is there. This is due to different namings here when the model is loaded.

Screenshot 2025-08-18 at 11 10 08 PM

The bottom 2 with simpler names are from currently loaded models, the top 2 are from all models. All 4 work correctly though.

@fbuechler
Copy link

@daniel-lxs I think the solution requires more complex matching logic because LM Studio seems to adapt the model keys to make them unique if you have different quantizations of the same model downloaded. In that case the key isn't necessarily a substring of the path any more.

Example:
unsloth/Qwen3-4B-Instruct-2507-GGUF/Qwen3-4B-Instruct-2507-UD-Q3_K_XL.gguf -> qwen3-4b-instruct-2507@q3_k_xl
unsloth/Qwen3-4B-Instruct-2507-GGUF/Qwen3-4B-Instruct-2507-UD-Q4_K_XL.gguf -> qwen3-4b-instruct-2507@q4_k_xl

@daniel-lxs
Copy link
Member Author

Then I think it would be great to create a properly scoped issue because lm-studio is making it quite hard for us to deduplicate the models.

@mechanicmuthu
@fbuechler

@fbuechler
Copy link

@daniel-lxs I'm sorry if I'm missing a point here but I don't think it needs to be that hard. In my opinion, the initial PR (#6956) provided a simple working solution that consistently used the model's path to prevent duplicate entries and replaced the downloaded model with more details if a loaded model with the same path was found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working lgtm This PR has been approved by a maintainer PR - Needs Review size:L This PR changes 100-499 lines, ignoring generated files.

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

LM Studio: Loaded models appear twice in Provider Configuration Profile

6 participants