Skip to content

Conversation

@giladgd
Copy link
Member

@giladgd giladgd commented Sep 27, 2025

Description of change

  • feat: Qwen3 Reranker support
  • fix: handle HuggingFace rate limit response
  • fix: adapt to llama.cpp breaking changes

The new Qwen3 Reranker support only works with recent models converted with llama.cpp release b6578 or later.
Here are prequantized models you can use right away:

Model URI Size
Qwen3 Reranker 0.6B hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q8_0 639.2MB
Qwen3 Reranker 4B hf:giladgd/Qwen3-Reranker-4B-GGUF:Q4_K_M 2.5GB
Qwen3 Reranker 8B hf:giladgd/Qwen3-Reranker-8B-GGUF:Q4_K_M 4.7GB

Pull-Request Checklist

  • Code is up-to-date with the master branch
  • npm run format to apply eslint formatting
  • npm run test passes with this change
  • This pull request links relevant issues as Fixes #0000
  • There are new or updated unit tests validating the change
  • Documentation has been updated to reflect this change
  • The new commits and pull request title follow conventions explained in pull request guidelines (PRs that do not follow this convention will not be merged)

@giladgd giladgd requested a review from ido-pluto September 27, 2025 12:59
@giladgd giladgd self-assigned this Sep 27, 2025
@giladgd giladgd enabled auto-merge (squash) September 27, 2025 19:04
Copy link
Contributor

@ido-pluto ido-pluto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@giladgd giladgd merged commit 00305f7 into master Sep 27, 2025
19 checks passed
@giladgd giladgd deleted the gilad/qwen3RerankSupport branch September 27, 2025 19:56
@github-actions
Copy link

github-actions bot commented Oct 2, 2025

🎉 This PR is included in version 3.14.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants