Search - excluded words, missing results, and multi-word queries

[bradumbaugh]

Hi folks,

Some questions/observations about search. Here's a search for when you're finished on the live MkDocs Material homepage, and the accompanying results:

This is an interesting case, because:

• when, a term in the query, isn't highlighted in the result
• Furthermore, the first result reports that when is missing, when it is not

A few related questions:

1. I assume that when is somehow explicitly excluded from the search—am I correct here? Is there a list of words that get removed from search queries? a / the / an / etc.? Is this configurable?

2. If I'm guessing right here—when is excluded from the search—would it be possible to prevent it from showing up as "missing" in the results? Perhaps it wouldn't make sense to do that, though... maybe there's something I'm not considering.

3. Related, but a bit of a tangent — I'm not sure I understand how the search operates on a query that contains a sequence of words separated by whitespace. Does it search for each word independently (and if so, will a document that matches more words rank higher than a document that matches fewer words), or does it look at the entire string? Are there modifiers like quotes (when you're finished vs "when you're finished") that change the behavior?

Thanks!

[squidfunk]

Thanks for creating this discussion!

1. Search is implemented with lunr.js which includes a stop word filter for most languages (see GitHub search on the lunr-languages repository). While this is configurable, Material for MkDocs, by default, adds the stopword filter to the search pipeline:

   mkdocs-material/src/partials/languages/en.html

   Line 38 in c96ff22

   "search.config.pipeline": "trimmer, stopWordFilter",

   You can override the behavior by using theme extension (see docs) and removing the stopWordFilter from the partial.

2. That should be possible. You can create an issue for filtering stopwords from the missing section.

3. Normally, lunr.js does an OR search and will rank documents, which contain all words higher. However, ranking is also influenced by factors such as the number of occurrences and document length. This can be configured in lunr.js. I try to provide sensible defaults but it's quite impossible to anticipate all cases. Search is always something that needs to be balanced and I think we have good defaults (we mostly use lunr's defaults). If you wish to tune the parameters, you could fork the theme and dive into lunr.js' internals.

   Another option would be to provide a custom transformation function and prefix all terms with +, which will turn it into an AND search. A good starting point is this section in the docs: https://squidfunk.github.io/mkdocs-material/setup/setting-up-site-search/#query-transformation

[bradumbaugh]

All makes sense. Thanks for the quick reply!

[bradumbaugh]

Filed issue #3343. Thanks again @squidfunk!