fix: resolve LM Studio context length detection (#5075) #5076
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixes #5075
This PR resolves the issue where LM Studio models incorrectly show a context length of "1" instead of their actual context window size.
Changes Made
fetchModel()method toLmStudioHandlerthat dynamically fetches model information from LM Studio using the existinggetLMStudioModels()functiongetModel()method to return actual model info with correct context length instead of static defaultsTesting
fetchModel()correctly callsgetLMStudioModels()getModel()returns actual model info after fetchingVerification of Acceptance Criteria
Technical Details
Root Cause: The
LmStudioHandler.getModel()method was returningopenAiModelInfoSaneDefaults(which hascontextWindow: 1) instead of fetching actual model information from LM Studio.Solution: The handler now dynamically fetches model information using the existing
getLMStudioModels()function, which correctly parses the actual context length from LM Studio's model metadata.Implementation Pattern: Follows the same pattern used by other providers (OpenRouter, Requesty) that implement a
fetchModel()method for dynamic model information retrieval.Files Changed
src/api/providers/lm-studio.ts- Added dynamic model fetching capabilitysrc/api/providers/__tests__/lmstudio.spec.ts- Updated tests to cover new functionalityChecklist
Important
Fixes context length detection in
LmStudioHandlerby dynamically fetching model info and enhancing error handling.LmStudioHandlernow dynamically fetches model info usingfetchModel()andgetLMStudioModels().getModel()returns actual model info with correct context length.lmstudio.spec.tsfor successful fetching, error scenarios, and backward compatibility.fetchModel()andgetModel()behavior.lm-studio.ts: Added dynamic model fetching and error handling.lmstudio.spec.ts: Updated tests to cover new functionality.This description was created by
for ac85aa6. You can customize this summary. It will automatically update as commits are pushed.