Improving search results

[day-jeff]

Search uses a logical OR on the terms of the query, and I've found that adding additional words INCREASES the number of matching documents rather than filtering it further.

I tried to fix this by implementing logical AND functionality, as described below. That said, I'm open to advice on other ways to solve the problem. For example, is there a way to configure the search results to sort documents so that those with all terms matching are listed first?

In pursuit of modifying all queries to logical AND, I had the notion of transforming search strings to requre each word ("+search +string"), but when it wasn't working, I simplified thing down to some test code to simply verify that I can modify the query:

{% extends "base.html" %}

{% block config %}
  {{ super() }}
  <script>
    var __search = {
      transform:function(query) {
        return "abc"
      }
    }
  </script>
{% endblock %}

This has no effect. Would be happy for advice on how to make this work, or on how to address the problem in a different way.

[squidfunk]

Search uses a logical OR on the terms of the query, and I've found that adding additional words INCREASES the number of matching documents rather than filtering it further.

Correct. This is the default behavior of Lunr.js, the search engine we use for implementation. The transform function was removed in 9.x. If you need it, you could downgrade to version 8.x. You might be interested in subscribing to this issue: #4980

We are also actively working on a new version of the search which will not exhibit this behavior! The next iteration will not be based on Lunr.js anymore and be much more tailored to documentation search. Here's a sneak peak.

[day-jeff]

Thx Martin. Do you have an ETA on the uodate?

[squidfunk]

An alpha version of the new search will be available in the coming weeks/months. There are so many open issues regarding feature requests for search right now, so it's a very high priority for us. Please understand that sometimes bugs and other change requests need to be fast tracked, but in general, the new search is what we're focusing on right now. I can't tell you the exact date, but I'll definitely announce it on Twitter and in the open change requests related to search that will be fixed with the next iteration.

[day-jeff]

Hi @squidfunk. Any update on your search improvements?

[squidfunk]

I'm currently actively working on it. I'm sorry this takes so long, but other stuff like the blog plugin with the 12k goal being reached got in the way. I'm confident that we can release the first version of the new iteration before the end of the year.

Please understand that this is a pretty big fish to fry, given the level of support we need for the base version alone (support for 66 languages, including Chinese and Japanese). We hope to raise more funding via sponsorships in the future, so I, personally, can delegate other tasks to others to focus on the core technology.

[squidfunk]

Please see the announcement in #6307 and the first research preview in #6321 – feedback wanted!