context mgt

Author: Serge H.

docs/roo-code-plus/configuration.md

@@ -33,4 +33,47 @@ The following settings have been added to support the LiteLLM provider. They are
 
 *   The `apiProvider` setting should be set to `"litellm"` to activate this provider.
 *   LiteLLM settings follow the same pattern as other providers, ensuring consistency.
-*   Existing configurations without LiteLLM settings remain valid and functional.
+*   Existing configurations without LiteLLM settings remain valid and functional.
+
+## OpenAI Compatible Provider Settings
+
+The following settings are relevant when `apiProvider` is set to `"openai"`.
+
+*   **`rooCode.apiConfiguration.openAiBaseUrl`** (`string`, optional):
+	*   The base URL of the OpenAI-compatible API endpoint.
+	*   Required. Example: `"https://api.mistral.ai/v1"`
+
+*   **`rooCode.apiConfiguration.openAiApiKey`** (`string`, optional):
+	*   The API key for the endpoint.
+	*   Stored securely in VS Code's SecretStorage.
+
+*   **`rooCode.apiConfiguration.openAiModelId`** (`string`, optional):
+	*   The specific model ID to use with the endpoint.
+	*   Example: `"mistral-large-latest"`, `"gpt-4o"`
+
+*   **`rooCode.apiConfiguration.openAiContextWindowOverride`** (`number`, optional):
+	*   Allows overriding the default context window size (128k tokens) assumed for compatible endpoints.
+	*   Set via a slider in the UI (Default, 8k, 32k, 128k, 512k, 1M, 2M tokens).
+	*   If set, this value is used for determining when to truncate conversation history. If unset (`undefined`), the default 128k is used.
+
+*   **`rooCode.apiConfiguration.openAiCustomModelInfo`** (`object`, optional):
+	*   Allows overriding other `ModelInfo` properties like `maxTokens`, `supportsImages`, pricing, etc., for the specified `openAiModelId`.
+	*   The `openAiContextWindowOverride` takes precedence over the `contextWindow` value within `openAiCustomModelInfo` if both are set.
+
+*   **`rooCode.apiConfiguration.openAiUseAzure`** (`boolean`, optional):
+	*   Set to `true` if connecting to an Azure OpenAI endpoint.
+
+*   **`rooCode.apiConfiguration.azureApiVersion`** (`string`, optional):
+	*   Specifies the API version for Azure OpenAI endpoints.
+
+*   **`rooCode.apiConfiguration.openAiHostHeader`** (`string`, optional):
+	*   Allows setting a custom `Host` HTTP header, sometimes required by proxy services.
+
+*   **`rooCode.apiConfiguration.openAiLegacyFormat`** (`boolean`, optional):
+	*   Set to `true` if the endpoint requires the older OpenAI message format (less common).
+
+*   **`rooCode.apiConfiguration.openAiR1FormatEnabled`** (`boolean`, optional):
+	*   Set to `true` for models requiring specific R1 formatting (e.g., DeepSeek Reasoner).
+
+*   **`rooCode.apiConfiguration.openAiStreamingEnabled`** (`boolean`, optional):
+	*   Defaults to `true`. Set to `false` if the endpoint does not support streaming responses.

docs/roo-code-plus/openai-context-override-plan.md

@@ -0,0 +1,58 @@
+# OpenAI Compatible Context Window Override Plan
+
+## Objective
+
+Allow users to configure the context window size for the "OpenAI Compatible" LLM provider type via the settings UI, overriding the default 128k limit.
+
+## Implementation Strategy
+
+Use a fixed-step slider UI element in the provider settings.
+
+## Plan Steps
+
+1.  **Schema Modification (`src/schemas/index.ts`):**
+    *   Add an optional property `openAiContextWindowOverride?: number;` to the relevant settings interface (likely `ProviderSettings` or a derived type). This field will store the user-selected token value (e.g., 8192, 32768) or `undefined` if the default is chosen.
+
+2.  **Backend Provider Logic Update (`src/api/providers/openai.ts`):**
+    *   Modify the `OpenAiHandler.getModel()` method.
+    *   Retrieve the base `ModelInfo` (from `openAiCustomModelInfo` or `openAiModelInfoSaneDefaults`).
+    *   If `this.options.openAiContextWindowOverride` is defined and valid, create a new `ModelInfo` object by copying the base info and setting `contextWindow` to the override value.
+    *   Return the potentially modified `ModelInfo`.
+
+3.  **UI Settings Update (Likely `webview-ui/src/components/Settings/ProviderSettings/OpenAISettings.tsx`):**
+    *   Add a `Slider` component with 7 discrete steps (0 to 6).
+    *   Map slider steps to token override values:
+        *   `0`: Default (sets override to `undefined`, effectively 128k)
+        *   `1`: 8k (sets override to `8192`)
+        *   `2`: 32k (sets override to `32768`)
+        *   `3`: 128k (sets override to `131072`)
+        *   `4`: 512k (sets override to `524288`)
+        *   `5`: 1M (sets override to `1048576`)
+        *   `6`: 2M (sets override to `2097152`)
+    *   Bind the slider's value to the `openAiContextWindowOverride` setting (mapping back and forth between the 0-6 scale and the token/undefined values).
+    *   Add a "Context Window Override" label.
+    *   Add a dynamic text display showing the selected token value (e.g., "Selected: 32k tokens", "Selected: Default (128k)").
+    *   Position this control after the model selection element.
+
+4.  **UI Chat Display Verification (Likely `webview-ui/src/components/Chat/ContextWindowProgress.tsx`):**
+    *   Ensure the component displaying the context size during chat correctly uses the `contextWindow` property from the `ModelInfo` object provided by the backend. No changes are expected if it already does this.
+
+5.  **Core Logic Verification (e.g., `src/core/sliding-window/index.ts`):**
+    *   Confirm that components like the sliding window manager consume the `contextWindow` value from the provider's `getModel().info` result. The override should propagate automatically.
+
+6.  **Internationalization (i18n):**
+    *   Add new keys to `webview-ui/src/i18n/locales/en/settings.json` for the label ("Context Window Override") and the dynamic display text.
+    *   Add placeholders or translations to other locale files as needed.
+  
+   ## Context Limit Usage
+  
+   *   **Verification Result:** Analysis of `src/core/sliding-window/index.ts` confirmed that the `contextWindow` value (including the override) is actively used by the `truncateConversationIfNeeded` function to determine when conversation history should be truncated before sending it to the API. It does not block requests but shortens the history based on the limit and buffer settings.
+  
+   ## Target Files
+
+*   `src/schemas/index.ts`
+*   `src/api/providers/openai.ts`
+*   `webview-ui/src/components/Settings/ProviderSettings/OpenAISettings.tsx` (or similar)
+*   `webview-ui/src/components/Chat/ContextWindowProgress.tsx` (for verification)
+*   `webview-ui/src/i18n/locales/en/settings.json`
+*   `webview-ui/src/i18n/locales/*/settings.json` (as needed)

src/api/providers/openai.ts

@@ -232,9 +232,24 @@ export class OpenAiHandler extends BaseProvider implements SingleCompletionHandl
 	}
 
 	override getModel(): { id: string; info: ModelInfo } {
+		// Get the base model info (custom or default)
+		const baseInfo = this.options.openAiCustomModelInfo ?? openAiModelInfoSaneDefaults
+
+		// Check for a valid override value
+		const overrideValue = this.options.openAiContextWindowOverride
+		let effectiveInfo = baseInfo
+
+		if (typeof overrideValue === "number" && overrideValue > 0) {
+			// Apply the override by creating a new object
+			effectiveInfo = {
+				...baseInfo,
+				contextWindow: overrideValue,
+			}
+		}
+
 		return {
 			id: this.options.openAiModelId ?? "",
-			info: this.options.openAiCustomModelInfo ?? openAiModelInfoSaneDefaults,
+			info: effectiveInfo, // Return the potentially overridden info
 		}
 	}
 

src/schemas/index.ts

@@ -347,6 +347,7 @@ export const providerSettingsSchema = z.object({
 	openAiUseAzure: z.boolean().optional(),
 	azureApiVersion: z.string().optional(),
 	openAiStreamingEnabled: z.boolean().optional(),
+	openAiContextWindowOverride: z.number().optional(),
 	// Ollama
 	ollamaModelId: z.string().optional(),
 	ollamaBaseUrl: z.string().optional(),

webview-ui/src/components/settings/ApiOptions.tsx

@@ -46,7 +46,17 @@ import {
 	useOpenRouterModelProviders,
 	OPENROUTER_DEFAULT_PROVIDER_NAME,
 } from "@/components/ui/hooks/useOpenRouterModelProviders"
-import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue, SelectSeparator, Button } from "@/components/ui"
+import {
+	Select,
+	SelectContent,
+	SelectItem,
+	SelectTrigger,
+	SelectValue,
+	SelectSeparator,
+	Button,
+	Slider,
+	Label,
+} from "@/components/ui"
 import { MODELS_BY_PROVIDER, PROVIDERS, VERTEX_REGIONS } from "./constants"
 import { AWS_REGIONS } from "../../../../src/shared/aws_regions"
 import { VSCodeButtonLink } from "../common/VSCodeButtonLink"
@@ -282,6 +292,37 @@ const ApiOptions = ({
 		}
 	}
 
+	// --- START: Helper functions and data for Context Window Slider ---
+	const contextWindowSteps = [
+		{ sliderValue: 0, overrideValue: undefined, label: t("settings:providers.contextWindow.default", { value: "128k" }) },
+		{ sliderValue: 1, overrideValue: 8192, label: "8k" },
+		{ sliderValue: 2, overrideValue: 32768, label: "32k" },
+		{ sliderValue: 3, overrideValue: 131072, label: "128k" },
+		{ sliderValue: 4, overrideValue: 524288, label: "512k" },
+		{ sliderValue: 5, overrideValue: 1048576, label: "1M" },
+		{ sliderValue: 6, overrideValue: 2097152, label: "2M" },
+	]
+
+	const mapOverrideToSliderValue = (override?: number): number => {
+		const step = contextWindowSteps.find((s) => s.overrideValue === override)
+		return step ? step.sliderValue : 0 // Default to step 0 if override is undefined or not found
+	}
+
+	const mapSliderValueToOverride = (sliderValue: number): number | undefined => {
+		const step = contextWindowSteps.find((s) => s.sliderValue === sliderValue)
+		return step ? step.overrideValue : undefined
+	}
+
+	const mapSliderValueToLabel = (sliderValue: number): string => {
+		const step = contextWindowSteps.find((s) => s.sliderValue === sliderValue)
+		return step ? step.label : "Unknown"
+	}
+
+	const currentSliderValue = mapOverrideToSliderValue(apiConfiguration?.openAiContextWindowOverride)
+	const currentSliderLabel = mapSliderValueToLabel(currentSliderValue)
+	// --- END: Helper functions and data for Context Window Slider ---
+
+
 	return (
 		<div className="flex flex-col gap-3">
 			<div className="flex flex-col gap-1 relative">
@@ -782,6 +823,33 @@ const ApiOptions = ({
 						serviceName="OpenAI"
 						serviceUrl="https://platform.openai.com"
 					/>
+					{/* START: OpenAI Context Window Override Slider */}
+					<div className="space-y-2">
+						<Label htmlFor="context-window-slider">
+							{t("settings:providers.contextWindow.overrideLabel")}
+						</Label>
+						<div className="flex items-center gap-3">
+							<Slider
+								id="context-window-slider"
+								min={0}
+								max={6}
+								step={1}
+								value={[currentSliderValue]}
+								onValueChange={(value) => {
+									const newOverride = mapSliderValueToOverride(value[0])
+									setApiConfigurationField("openAiContextWindowOverride", newOverride)
+								}}
+								className="flex-grow"
+							/>
+							<span className="text-sm text-vscode-descriptionForeground min-w-[100px] text-right">
+								{t("settings:providers.contextWindow.selectedValue", { value: currentSliderLabel })}
+							</span>
+						</div>
+						<div className="text-sm text-vscode-descriptionForeground">
+							{t("settings:providers.contextWindow.description")}
+						</div>
+					</div>
+					{/* END: OpenAI Context Window Override Slider */}
 					<R1FormatSetting
 						onChange={handleInputChange("openAiR1FormatEnabled", noTransform)}
 						openAiR1FormatEnabled={apiConfiguration?.openAiR1FormatEnabled ?? false}

webview-ui/src/i18n/locales/en/settings.json

@@ -150,6 +150,12 @@
 		"googleCloudKeyFile": "Google Cloud Key File Path",
 		"googleCloudProjectId": "Google Cloud Project ID",
 		"googleCloudRegion": "Google Cloud Region",
+		"contextWindow": {
+			"overrideLabel": "Context Window Override",
+			"selectedValue": "Selected: {{value}}",
+			"default": "Default ({{value}})",
+			"description": "Override the default context window size for this OpenAI compatible endpoint. 'Default' uses 128k tokens."
+		},
 		"lmStudio": {
 			"baseUrl": "Base URL (optional)",
 			"modelId": "Model ID",