fix: integrate LmStudioHandler with centralized model cache (#5075) #5081
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixes #5075
This PR fixes the issue where LM Studio models were showing context length "1" instead of their actual values. The problem was that
LmStudioHandler.getModel()was returning static defaults instead of using dynamically fetched model information.Changes Made
LmStudioHandler.fetchModel()to usegetModels({ provider: "lmstudio" })from the model cache instead of callinggetLMStudioModels()directlygetModelWithFetch()method and updatedcreateMessage()andcompletePrompt()to useawait this.fetchModel()directlyRequestyHandlerand other providers that use the centralized model cachegetModelsinstead ofgetLMStudioModelsand updated expectations to match the new implementationTesting
Verification of Acceptance Criteria
Checklist
Files Changed
src/api/providers/lm-studio.ts- Updated to use centralized model cachesrc/api/providers/__tests__/lmstudio.spec.ts- Updated tests to match new implementationHow it works
Before:
getModel()returnedopenAiModelInfoSaneDefaultswith context length 128,000 but maxTokens -1After:
fetchModel()retrieves actual model information from LM Studio via the model cache, including the correct context length from the model's metadataThis ensures that users will see the actual context length of their LM Studio models instead of the generic default value.