Add support for Simple ReRanker or similar locally hosted version

[kalaspuffar]

I've been playing around with locally hosting a version of LibreChat, and to get it really useful, a web search feature was a requirement. I've hosted SearXNG and FireCrawl locally. Also ran Jina to do the reranking. But I could not find a locally hosted open source solution for reranking. So I set out to create one.

I found this repository:
https://github.com/AnswerDotAI/rerankers

And created a small wrapper Docker image api for it:
https://github.com/kalaspuffar/simple-reranker

It should pretty much be a slot-in solution alongside Jina and Cohere.

This was the code I tried with and never got to work:

export class SimpleReranker extends BaseReranker {
  constructor({
    instanceUrl = process.env.SIMPLE_RERANKER_HOST,
    logger,
  }: {
    instanceUrl?: string;
    logger?: t.Logger;
  }) {
    super(logger);
    this.instanceUrl = instanceUrl;
  }

  async rerank(
    query: string,
    documents: string[],
    topK: number = 5
  ): Promise<t.Highlight[]> {
    this.logger.debug(`Reranking ${documents.length} chunks with SimpleReranker`);

    if (this.instanceUrl == null || this.instanceUrl === '') {
      this.logger.warn('SIMPLE_RERANKER_HOST is not set. Using default ranking.');
      return this.getDefaultRanking(documents, topK);
    }

    try {
      const requestData = {
        config: {
          model_name: process.env.SIMPLE_RERANKER_MODEL_NAME,
          model_type: process.env.SIMPLE_RERANKER_MODEL_TYPE,
          lang: process.env.SIMPLE_RERANKER_LANG,
          api_provider: process.env.SIMPLE_RERANKER_API_PROVIDER,
          api_key: process.env.SIMPLE_RERANKER_API_KEY
        },
        data: {
          query: query,
          docs: documents,
          top_k: topK,
        }
      };

      return await axios.post(
        this.instanceUrl,
        requestData,
        {
          headers: {
            'Content-Type': 'application/json',
          },
        }
      );

    } catch (error) {
      this.logger.error('Error using Simple reranker:', error);
      // Fallback to default ranking on error
      return this.getDefaultRanking(documents, topK);
    }
  }
}

I'm willing to help out in the development process and share any of my work with the community. What you have done so far is great, and I'm looking forward to working with you.

Best regards
Daniel

[danny-avila]

Thanks @kalaspuffar we were thinking of adding something like to the RAG API, feel free to make a PR there:
https://github.com/danny-avila/rag_api