feat: Simple ReRanker local models.

[kalaspuffar]

This is a PR as per the suggestion from danny-avila/LibreChat#9102

This will add an endpoint /rerank in order to use open source models to rerank documents. The endpoint needs a query to rerank against and documents to rank. We can also add information on how many results we need, k, and a configuration to set the model and keys in order to run this operation.

All available configuration options could be found over at https://github.com/AnswerDotAI/rerankers, which this endpoint is a thin wrapper over.

Test call

curl -s http://localhost:8000/rerank \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer YOUR_JWT_TOKEN' \
  -d '{
    "query": "I love you",
    "docs": ["I hate you", "I really like you"],
    "k": 5
  }'

Expected response:

[{"text":"I really like you","score":-1.537894606590271},{"text":"I hate you","score":-4.30911111831665}]

Realized that sending the model over the call is not the correct option, we need to load it one time to improve performance so now you can configure that in the environment for the rag_api repository.

SIMPLE_RERANKER_MODEL_NAME = "mixedbread-ai/mxbai-rerank-large-v1"
SIMPLE_RERANKER_MODEL_TYPE = "cross-encoder"
#SIMPLE_RERANKER_MODEL_NAME = "ms-marco-MiniLM-L-12-v2"
#SIMPLE_RERANKER_MODEL_NAME = "flashrank"
#SIMPLE_RERANKER_MODEL_TYPE = "colbert"
SIMPLE_RERANKER_LANG = ""
SIMPLE_RERANKER_API_PROVIDER = ""
SIMPLE_RERANKER_API_KEY = ""

[kalaspuffar]

Force push was due to black linting.

All done! ✨ 🍰 ✨
1 file reformatted, 1 file left unchanged.

[Copilot]

Pull request overview

This PR adds a new /rerank endpoint to enable document reranking using open source models via the rerankers library. The implementation allows users to submit a query and a list of documents to be reranked based on relevance, with optional control over the number of top results returned.

Key Changes:

• Added rerankers library dependencies with transformers and flashrank support
• Implemented /rerank endpoint that accepts queries and documents for reranking
• Configured Docker Compose with NVIDIA runtime and HuggingFace cache volume for model support

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 8 comments.

File	Description
requirements.txt	Added rerankers library with transformers and flashrank extras for document reranking functionality
docker-compose.yaml	Added NVIDIA runtime support and HuggingFace cache volume mount to support GPU-accelerated model inference
app/routes/document_routes.py	Implemented reranker instance initialization and /rerank endpoint handler with document processing logic
app/models.py	Added QueryMultipleDocs Pydantic model to define request schema for the rerank endpoint

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.