MeiliSearch call error when posting the first message in a chat

[ykare]

What happened?

When posting the first message to a chat, an error occurs during a MeiliSearch call. No error is displayed in the UI, but the error messages are output to Meilisearch-related logs.

This issue seems to occur because the system is trying to retrieve the index of a Conversation that does not yet exist in MeiliSearch when posting the first message in a chat.

The following fix has been confirmed to improve the situation:

--- a/api/models/plugins/mongoMeili.js
+++ b/api/models/plugins/mongoMeili.js
@@ -346,7 +346,8 @@ module.exports = function mongoMeili(schema, options) {
     // Doc is a Conversation
     if (doc.messages) {
       try {
-        meiliDoc = await client.index('convos').getDocument(doc.conversationId);
+        if (doc._meiliIndex)
+          meiliDoc = await client.index('convos').getDocument(doc.conversationId);
       } catch (error) {
         logger.debug(
           '[MeiliMongooseModel.findOneAndUpdate] Convo not found in MeiliSearch and will index ' +

I appreciate your attention to this issue.

Steps to Reproduce

1. Create a new chat.
2. Post the first message.
3. docker compose logs -f meilisearch
4. show logs/meiliSync-YYYY-MM-DD.log

What browsers are you seeing the problem on?

Chrome, Microsoft Edge

Relevant log output

docker compose logs:
chat-meilisearch  | 2024-08-20T04:56:09.868787Z  WARN HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/XXXXXXXXX query_parameters= user_agent=node status_code=404 error=Document `d23b64c1-5f44-4f84-bd7e-75ee4fc1c47d` not found.}: tracing_actix_web::middleware: Error encountered while processing the incoming HTTP request: ResponseError { code: 404, message: "Document `d23b64c1-5f44-4f84-bd7e-75ee4fc1c47d` not found.", error_code: "document_not_found", error_type: "invalid_request", error_link: "https://docs.meilisearch.com/errors#document_not_found" }
chat-meilisearch  | 2024-08-20T05:36:01.630200Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/XXXXXXXXX query_parameters= user_agent=node status_code=404 error=Document `38cdbd3e-fa68-43be-9e74-8b890a206468` not found.}: meilisearch: close time.busy=359µs time.idle=352µs

meiliSync-YYYY-MM-DD.log:
{"code":"document_not_found","httpStatus":404,"level":"debug","link":"https://docs.meilisearch.com/errors#document_not_found","message":"[MeiliMongooseModel.findOneAndUpdate] Convo not found in MeiliSearch and will index d23b64c1-5f44-4f84-bd7e-75ee4fc1c47d Document `d23b64c1-5f44-4f84-bd7e-75ee4fc1c47d` not found.","name":"MeiliSearchApiError","stack":"MeiliSearchApiError: Document `d23b64c1-5f44-4f84-bd7e-75ee4fc1c47d` not found.\n    at /app/node_modules/meilisearch/dist/bundles/meilisearch.cjs.js:378:19\n    at Generator.next (<anonymous>)\n    at fulfilled (/app/node_modules/meilisearch/dist/bundles/meilisearch.cjs.js:298:58)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)","type":"invalid_request"}

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Hi @ykare this is by design so I moved to discussion.

if (doc._meiliIndex)

It's possible the document may not have this property and yet the conversation may exist in meili, this was my thinking, also querying the database wouldn't be good here in my opinion

[ykare]

Hi @danny-avila ,Thanks for your quick reply!

I see, so there's a possibility that the Meili index exists even without the doc._meiliIndex property. I have a few questions about this:

1. Is the Meili index still created when doc.postSaveHook() is called in schema.post('findOneAndUpdate', ...) in mongoMeili.js, as I previously understood?

2. The postSaveHook() looks like this, right?

postSaveHook() {
    if (this._meiliIndex) {
        this.updateObjectToMeili();
    } else {
        this.addObjectToMeili();
    }
}

3. I thought addObjectToMeili() would add to the Meili index, set doc._meiliIndex to true, and save to MongoDB. Is this not the case?

But from your explanation, it sounds like even if doc._meiliIndex doesn't exist when findOneAndUpdate is called on a Conversion object, the Meili index might still exist, is that right?

Could you explain a bit more about the benefits of this design and in what cases this situation might occur? Also, I'm curious about how you manage the synchronization between the Meili index and the _meiliIndex property with this design. Can you shed some light on that?

[homjay]

Is there any solution? I have encountered this problem as well. The AI PDF and MeiliSearch-related method do not work, but there are no solutions at the moment.

[danny-avila]

What error are you experiencing exactly? Is it causing any issues with the app?