Add wildcard analyzer

[MBkkt]

• Analyzer description
• View examples
• Inverted index examples
• Explain how the search works internally?
• Release notes

Upstream PRs

• 3.10:
• 3.11:
• 3.12: arangodb/enterprise-preview:devel-nightly

[arangodb-docs-automation]

Deploy Preview Available Via
https://deploy-preview-392--docs-hugo.netlify.app

[Simran-B]

The Analyzer seems to work differently than the one in Elastic, at least based on what they show in their blog post. They don't seem to create all of the n-grams, e.g. for avocado and n-gram size = 3, they would apparently add avo, oca, ado to the index but our Analyzer returns:

av, avo, voc, oca, cad, ado, do, o.

The first token seems to always be a prefix with ngramSize - 1? Then all the trigrams, and I suppose the rest of the strings down to a size of 1 are for suffix matching. I don't quite understand the first entry. If ngramSize is e.g. 5 but the input is shorter than this, the first two entries are identical. Is this intentional?

[MBkkt]

@Simran-B

It returns all ngram with specified ngramSize, and all suffix ngrams smaller than ngramSize

It also change input text to the \xFFtext\xFF -- \xFF invalid utf-8 and cannot appear in any text, so it real output is
\xFFav, avo, voc, oca, cad, ado, do\xFF, o\xFF

This is necessary to speedup queries with max sub pattern size smaller than ngramSize and prefix/suffix queries

[MBkkt]

@Simran-B

The Analyzer seems to work differently than the one in Elastic, at least based on what they show in their blog post. They don't seem to create all of the n-grams, e.g. for avocado and n-gram size = 3, they would apparently add avo, oca, ado to the index but our Analyzer returns:

It's just simplification of blog-post
If analyzer will produce for avocado only avo oca ado.
It will imposible to fast find it with query like %cad%

But it's possible on search phase avoid all tokens which intersects, except first and last ngram
So if you honestly ngram avocado, it's possible to search avocado with \xFFav oca do\xFF, and you don't need to search avo, voc, cad, ado
It can be better in some cases, but from our measurements commonly it slower (because smaller count of distinct terms in approximation query)

[MBkkt]

lgtm

[Simran-B]

/generate

[Simran-B]

/generate

[Simran-B]

The API change described in #447 broke an example. The example erroneously tried to drop a collection that is part of a graph but dropping the example graph drops all graph collections anyway.

Another issue that surfaced is that curl examples that purposefully trigger an error need to make use of // xpError(...) or the new toolchain complains about an unexpected error. In the old toolchain, the correct example behavior was only ensured by assert() statements.

[Simran-B]

/commit

[nerpaula]

LGTM