Use a fulltext index for search

[AlexeyBond]

Currently there is a case-insensitive regular expression applied to some fields to search for documents in admin panel (when using search bar or searching for document in relation field). That's ok for small collections.
However, it's inefficient in case of large enough collections since mongodb cannot utilize indexes for such queries. So that warning is most likely useless.

Short sad story about regex queries

We are currently using Strapi and we have some collections with from dozens of thousands to few millions documents. When a moderator enters a text in search bar strapi sends a query with regular expressions, like the ones Payload does. Except it automatically generates query that tries to match EVERY known text or numeric field against the regular expression.

So, there were few accidents when moderators typed query in search bar multiple times, requests were sent to database and left hanging there consuming all available CPU for few hours.
Fortunately, mongodb is good enough at concurrency, so other users experienced just a little degradation of performance in most cases.

The search can be much more efficient if implemented using a fulltext index.
The fulltext index should be created for collection and some sort of special query should be sent instead of standard query with "like" operators for each searchable field.

I guess, the choice to use a fulltext index for search should be left optional since it's an overkill for small collections.
Also, it makes sense to allow some customization of full-text search functionality to make it possible to use Atlas Search, external Elasticsearch server or something else:

const MyCollection: CollectionConfig = {
  ...,
  searchEngine:
    (_: { schema: mongoose.Schema, collectionConfig: CollectionConfig}) =>
      async (_: { query: string, collection: payload.Collection }): Document[] => { ... }
};

sanitizeConfig may substitute a default implementation (using regex queries) of searchEngine if no searchEngine is provided.
Another implementation may use a fulltext index, adding it to the schema when invoked first time and performing a fulltext query on invocation of returned function.
But maybe that's a little over-engineered suggestion.

[finkinfridom]

Generally speaking, it would be ideal to have the possibility to:

• configure deeper the searchEngine or
• substitute the searchEngine with a different implementation/service

this will help, especially in the millions of documents scenario.