fix: prevent LM Studio context overflow by limiting maxTokens to 20% of context window

[roomote]

Summary

This PR fixes the LM Studio crash issue when the context reaches 128K tokens despite having a 200K limit configured.

Problem

Users were experiencing crashes with LM Studio when the context reached 128K tokens, even though they had configured a 200K context limit. The error message was:

The model has crashed without additional information. (Exit code: 18446744072635812000)

Root Cause

The issue was in the parseLMStudioModel function where maxTokens (maximum output tokens) was being set equal to contextWindow. This meant the model was trying to reserve the entire context window for output, leaving no room for input tokens, causing an overflow.

Solution

• Calculate maxTokens as 20% of the contextWindow instead of using the full context size
• This ensures there's always sufficient room for input tokens (80% of context)
• Follows the same pattern already used by other providers in the codebase

Changes

• Updated parseLMStudioModel function to calculate maxTokens = Math.ceil(contextLength * 0.2)
• Added comprehensive test coverage for the new calculation logic
• Added clear comments explaining the rationale

Testing

• All existing tests pass ✅
• Added new test cases covering various context window sizes
• Linting and type checking pass ✅

Fixes #7388

---

Important

Fixes LM Studio crash by setting maxTokens to 20% of contextWindow in parseLMStudioModel, ensuring space for input tokens.

• Behavior:
  • Fixes crash in LM Studio by setting maxTokens to 20% of contextWindow in parseLMStudioModel.
  • Ensures input tokens have sufficient space (80% of context).
• Testing:
  • Adds test cases in lmstudio.test.ts for various contextLength values to verify maxTokens calculation.
  • All existing tests pass.
• Misc:
  • Adds comments in lmstudio.ts explaining maxTokens calculation rationale.

^{This description was created by for 8119af6. You can customize this summary. It will automatically update as commits are pushed.}

[roomote]

Reviewing my own code because apparently I trust no one, not even myself.

[roomote]

Could we consider making this ratio configurable? While 20% is a reasonable default that matches other providers, some users might want to adjust this based on their specific use cases. Perhaps a setting like lmstudio.maxOutputRatio with a default of 0.2?

[roomote]

For models with very small context windows (e.g., < 1000 tokens), this 20% calculation might result in very limited output capacity. Should we consider adding a minimum threshold? Something like:

Suggested change

	const maxOutputTokens = Math.ceil(contextLength * 0.2)
	// Calculate maxTokens as 20% of context window to prevent context overflow
	// This ensures there's always room for input tokens and prevents crashes
	// when approaching the context limit
	const calculatedMaxTokens = Math.ceil(contextLength * 0.2)
	// Ensure a minimum of 200 tokens for very small context windows
	const maxOutputTokens = Math.max(calculatedMaxTokens, Math.min(200, contextLength))

[roomote]

Great test coverage! The test cases cover a good range of context sizes. Consider also adding a test case for very small context windows (e.g., 512 tokens) to ensure the calculation works correctly at the lower bounds.

[akierum]

So when to expect this to be merged in next update?

[akierum]

So when to expect this to be merged in next update?