Skip to content

fix(litellm): handle baseurl with paths correctly #5697

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
mrubens merged 2 commits into from
Jul 15, 2025
Merged

fix(litellm): handle baseurl with paths correctly #5697

mrubens merged 2 commits into from
Jul 15, 2025

Conversation

@ChuKhaLi
Copy link
Contributor

@ChuKhaLi ChuKhaLi commented Jul 14, 2025 •
edited by ellipsis-dev bot
Loading

Related GitHub Issue

Closes: #5696

Roo Code Task Context (Optional)

https://app.roocode.com/share/5ae63ae4-aae9-4a92-8dae-36b9bc5f550d

Description

This PR fixes a bug where the path component of the baseURL for the LiteLLM provider contains path in it

Test Procedure

  1. Configure the litellm provider in VSCode settings with a baseURL that includes a path, for example http://localhost:4000/litellm
  2. Click Refresh Models
  3. It should fetch the latest model from Lite LMM instance
  4. try to send a message to Roo Code

Pre-Submission Checklist

  • Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above).
  • Scope: My changes are focused on the linked issue (one major feature/fix per PR).
  • Self-Review: I have performed a thorough self-review of my code.
  • Testing: New and/or updated tests have been added to cover my changes (if applicable).
  • Documentation Impact: I have considered if my changes require documentation updates (see "Documentation Updates" section below).
  • Contribution Guidelines: I have read and agree to the Contributor Guidelines.

Screenshots / Videos

Documentation Updates

Additional Notes

Get in Touch

Important

Fixes URL handling in getLiteLLMModels to correctly process base URLs with paths and trailing slashes, with new test cases added.

  • Behavior:
    • Fixes URL construction in getLiteLLMModels in litellm.ts to handle base URLs with paths and trailing slashes correctly.
    • Uses URL constructor to ensure proper URL joining.
  • Tests:
    • Adds test cases in litellm.spec.ts for base URLs with paths and trailing slashes.
    • Verifies correct URL formation and request handling with mock responses.

This description was created by Ellipsis for ed51aef. You can customize this summary. It will automatically update as commits are pushed.

@ChuKhaLi ChuKhaLi requested review from cte, jr and mrubens as code owners July 14, 2025 16:35
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. bug Something isn't working labels Jul 14, 2025
@ChuKhaLi ChuKhaLi mentioned this pull request Jul 14, 2025
Closed
@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Jul 14, 2025
@daniel-lxs daniel-lxs moved this from Triage to renovate BOT in Roo Code Roadmap Jul 15, 2025
@daniel-lxs daniel-lxs moved this from renovate BOT to PR [Needs Prelim Review] in Roo Code Roadmap Jul 15, 2025
@hannesrudolph hannesrudolph added PR - Needs Preliminary Review and removed Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. labels Jul 15, 2025
@daniel-lxs
fix: improve URL construction robustness and add edge case tests
c2a0446 
- Use URL object manipulation for more robust path joining
- Handle edge cases: double slashes, query params, fragments
- Normalize paths to prevent duplicate slashes
- Add comprehensive test coverage for all URL scenarios
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Jul 15, 2025
daniel-lxs
daniel-lxs approved these changes Jul 15, 2025
View reviewed changes
Copy link
Member

@daniel-lxs daniel-lxs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @ChuKhaLi!

I made some small tweaks to the pathname normalization.

It looks good to me!

@daniel-lxs daniel-lxs moved this from PR [Needs Prelim Review] to PR [Needs Review] in Roo Code Roadmap Jul 15, 2025
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Jul 15, 2025
@mrubens mrubens merged commit 9b6fb36 into RooCodeInc:main Jul 15, 2025
20 checks passed
@github-project-automation github-project-automation bot moved this from PR [Needs Review] to Done in Roo Code Roadmap Jul 15, 2025
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Jul 15, 2025
fxcl added a commit to tameslabs/Roo-Cline that referenced this pull request Jul 16, 2025
@fxcl
Merge branch 'main' into gemini-cli
089c425 
* main:
  fix: Resolve confusing auto-approve checkbox states (RooCodeInc#5602)
  fix: prevent empty mode names from being saved (RooCodeInc#5766) (RooCodeInc#5794)
  Format time in ISO 8601 (RooCodeInc#5793)
  fix: resolve DirectoryScanner memory leak and improve file limit handling (RooCodeInc#5785)
  Fix settings dirty check (RooCodeInc#5779)
  feat: increase Ollama API timeout values and extract as constants (RooCodeInc#5778)
  fix: Exclude Terraform and Terragrunt cache directories from checkpoints (RooCodeInc#4601) (RooCodeInc#5750)
  Move less commonly used provider settings into an advanced dropdown (RooCodeInc#5762)
  feat: Add configurable error & repetition limit with unified control (RooCodeInc#5654) (RooCodeInc#5752)
  list-files must include at least the first-level directory contents (RooCodeInc#5303)
  Update evals repo link (RooCodeInc#5758)
  Feature/vertex ai model name conversion (RooCodeInc#5728)
  fix(litellm): handle baseurl with paths correctly (RooCodeInc#5697)
  Add telemetry for todos (RooCodeInc#5746)
  feat: add undo functionality for enhance prompt feature (fixes RooCodeInc#5741) (RooCodeInc#5742)
  Fix max_tokens limit for moonshotai/kimi-k2-instruct on Groq (RooCodeInc#5740)
  Changeset version bump (RooCodeInc#5735)
  Add changeset for v3.23.12 patch release (RooCodeInc#5734)
  Update the max-token calculation in model-params to use the shared logic (RooCodeInc#5720)
  Changeset version bump (RooCodeInc#5719)
  chore: add changeset for v3.23.11 patch release (RooCodeInc#5718)
  Add Kimi K2 model and better support (RooCodeInc#5717)
  Fix: Remove invalid skip-checkout parameter from GitHub Actions workflows (RooCodeInc#5676)
  feat: add Cmd+Shift+. keyboard shortcut for previous mode switching (RooCodeInc#5695)
  Changeset version bump (RooCodeInc#5708)
  chore: add changeset for v3.23.10 patch release (RooCodeInc#5707)
  Add padding to the index model options (RooCodeInc#5706)
  fix: prioritize built-in model dimensions over custom dimensions (RooCodeInc#5705)
  Update CHANGELOG.md
  Changeset version bump (RooCodeInc#5702)
  chore: add changeset for v3.23.9 patch release (RooCodeInc#5701)
  Tweaks to command timeout error (RooCodeInc#5700)
  Update contributors list (RooCodeInc#5639)
  feat: enable Claude Code provider to run natively on Windows (RooCodeInc#5615)
  feat: Add configurable timeout for command execution (RooCodeInc#5668)
  feat: add gemini-embedding-001 model to code-index service (RooCodeInc#5698)
  fix: resolve vector dimension mismatch error when switching embedding models (RooCodeInc#5616) (RooCodeInc#5617)
  fix: [5424] return the cwd in the exec tool's response so that the model is not lost after subsequent calls (RooCodeInc#5667)
  Changeset version bump (RooCodeInc#5670)
  chore: add changeset for v3.23.8 patch release (RooCodeInc#5669)
roomote bot pushed a commit that referenced this pull request Aug 22, 2025
@roomote
feat: add prompt caching support for Groq provider
34abaf0 
- Enable supportsPromptCache flag for all Groq models
- Add cacheReadsPrice with 80% discount on cached tokens
- Override createMessage to handle Groq cache metrics from prompt_tokens_details
- Update tests to verify cache token handling
- Similar implementation to Cline PR #5697
roomote bot pushed a commit that referenced this pull request Aug 22, 2025
@roomote
feat: add comprehensive prompt caching support for Groq provider
593d9ed 
- Enable supportsPromptCache flag for all Groq models with 80% discount pricing
- Add groqUsePromptCache setting to enable/disable caching
- Implement GroqCacheStrategy for optimal message formatting
- Override createMessage to handle multiple cache token field names
- Add conversation cache state management
- Add comprehensive test coverage for caching functionality

Similar to Cline PR #5697 but adapted for Groq automatic prefix caching
@roomote roomote bot mentioned this pull request Aug 22, 2025
Closed
daniel-lxs added a commit that referenced this pull request Aug 22, 2025
@daniel-lxs
feat: add prompt caching support for Kimi K2 on Groq
8fa6f00 
Ported from upstream Cline repository PR #5697
Original PR: cline/cline#5697

- Added GroqUsage interface to handle cached token fields
- Implemented proper cost calculation with cache read discounts
- Enabled prompt caching for Kimi K2 model with 50% discount on cached tokens
- Updated tests to verify caching functionality

Co-authored-by: Cline Contributors <[email protected]>
mrubens pushed a commit that referenced this pull request Aug 22, 2025
@daniel-lxs
feat: add prompt caching support for Kimi K2 on Groq (#7324)
faab314 
Ported from upstream Cline repository PR #5697
Original PR: cline/cline#5697

- Added GroqUsage interface to handle cached token fields
- Implemented proper cost calculation with cache read discounts
- Enabled prompt caching for Kimi K2 model with 50% discount on cached tokens
- Updated tests to verify caching functionality

Co-authored-by: Cline Contributors <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mrubens mrubens mrubens approved these changes

@daniel-lxs daniel-lxs daniel-lxs approved these changes

@cte cte Awaiting requested review from cte cte is a code owner

@jr jr Awaiting requested review from jr jr is a code owner

Assignees

No one assigned

Labels

bug Something isn't working lgtm This PR has been approved by a maintainer PR - Needs Review size:L This PR changes 100-499 lines, ignoring generated files.

Projects

Roo Code Roadmap
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

LiteLLM - Incorrect baseURL handling in LiteLLM provider when a path is included

4 participants

@ChuKhaLi @mrubens @daniel-lxs @hannesrudolph