feat: Add provider filtering support to router models backend #8916
Conversation
Allows frontend to request specific subset of router models instead of fetching all providers. This significantly reduces payload sizes and memory usage when only specific providers are needed. - Honor message.values.providers filter in requestRouterModels handler - Fetch only requested providers when filter is present - Maintain backward compatibility with existing aggregate behavior - Add comprehensive test coverage for filtering logic
dosubot
bot
added
size:L
Review Complete ✅No issues found. The implementation is clean, type-safe, and well-tested. What Was Verified✅ Type Safety: Filter parsing uses proper type guards and error handling to safely convert strings to ✅ Backward Compatibility: When no filter is provided, all providers are fetched (maintains existing behavior) ✅ Edge Cases: Handles null/undefined, empty arrays, invalid provider names gracefully ✅ Test Coverage: Comprehensive tests verify filtered, unfiltered, and single-provider scenarios ✅ Error Handling: Failed provider fetches are caught via ✅ Code Quality: Follows existing patterns and coding standards |
Implements intelligent provider filtering in the frontend to request only the models needed for the currently selected provider. This dramatically reduces memory usage by avoiding unnecessary fetches and smaller cache footprints. - Add DYNAMIC_ROUTER_PROVIDERS set to identify providers needing models - Disable useRouterModels for static providers (anthropic, openai-native, etc.) - Pass provider-specific filters for dynamic providers - Add observability logging to track request/response patterns - Update tests for new behavior Works with backend filtering (PR #8916) to achieve end-to-end payload reduction.
![roomote[bot]](https://avatars.githubusercontent.com/in/1546624?s=60&v=4){kind=small}
hannesrudolph
added
the
Issue/PR - Triage
hannesrudolph
added
PR - Needs Preliminary Review
and removed
Issue/PR - Triage
…er filter to avoid race with active-only payload (pairs with #8916 backend filtering)
daniel-lxs
moved this from PR [Needs Prelim Review]
to PR [Needs Review]
in Roo Code Roadmap
| roo: {}, | ||
| } | ||
| // Optional providers filter coming from the webview | ||
| const providersFilterRaw = Array.isArray(message?.values?.providers) ? message.values.providers : undefined |
There was a problem hiding this comment.
Is there a reason for this filter to be an array instead of a single provider?
There was a problem hiding this comment.
In general, if we only want to fetch one at a time I think we should make a function that only fetches one and use that instead of having the conditional logic. The only reason for backwards compatibility is that we broke this into multiple PRs, right? That seems ok but I think we should make a new method for the single lookup and then delete the old one once the webview is updated. Or just do it all in one PR, idk
- Simplify backend to use single provider filter instead of array - Use existing isDynamicProvider from @roo-code/types instead of duplicating - Update frontend to request only the needed provider - Gate router model fetches entirely for static providers - Remove unnecessary debug logging - Update tests to reflect new behavior This combines and improves upon PRs #8916 and #8917 by: 1. Using a simpler single-provider API (provider: string instead of providers: string[]) 2. Leveraging existing type definitions from packages/types 3. Eliminating race conditions through simpler request/response matching 4. Reducing payload sizes and network usage Closes #8916 Closes #8917
|
Closing in favor of combined PR #8956 which addresses all feedback and uses a simpler single-provider API |
github-project-automation
bot
moved this from PR [Needs Review]
to Done
in Roo Code Roadmap
Problem
Part of the webview memory mitigation effort. The backend was always fetching and returning all router models from all providers, even when the frontend only needed models from a specific provider. This resulted in unnecessarily large payloads being cached and reconciled in the webview.
Solution
Add support for provider filtering in the
requestRouterModelshandler. The frontend can now send aprovidersfilter viamessage.values.providers, and the backend will only fetch and return models for those specific providers.Changes
message.values.providersfilterImpact
Before: Backend always returned all router models (openrouter, roo, litellm, etc.) regardless of which provider was needed
After: Backend can return filtered subset (e.g., only Roo models when
providers: ["roo"]is requested)Payload size reduction: Instead of 8+ provider catalogs, can return just 1
Files Changed
src/core/webview/webviewMessageHandler.tssrc/core/webview/__tests__/webviewMessageHandler.routerModels.spec.ts(new)Testing
Part of
This is the second PR in a series to reduce webview memory usage:
Dependencies
This PR is independent and can be merged before or after #8915. However, it enables the frontend filtering in PR #3 to work effectively.
Important
Adds provider filtering to
webviewMessageHandlerforrequestRouterModels, reducing payload size by fetching only specified providers.webviewMessageHandlernow supports provider filtering forrequestRouterModelsusingmessage.values.providers.webviewMessageHandler.routerModels.spec.tsadded.This description was created by
for 0354924. You can customize this summary. It will automatically update as commits are pushed.