SQLite unable to get context on Wiki with more than > 2.400 pages

[eduardomozart]

Issue Description

Hello splitbrain,
Thank you for your plugin. I have two wikis: one with 1.500 pages and another with 2.462 pages. Not sure if it's related to the number of pages or the length of it's contents. The one with 2.462 pages generated a SQLite file with about ~250 MB, and the other one has ~150 MB. Both WIkis contains the same contents, the difference is that some namespaces aren't synced between them.
Both are using the same engines:

The 1.500 pages Wiki generates responses using SQLite as expected (with some warnings about "slow query" on SQLite plugin at DokuWiki log viewer), but the other one, doesn't.
Debugging the ''helper.log'' file, it seems that it doesn't find any context on the larger Wiki to the same question ("O que é o ClearPass?" is the question), but both have the "clearpass" namespace synced with the same contents.
I believe it's related to the SQLite because when I changed the Storage to Qdrant both Wiki starts working as expected.
I can provide the SQLite and the data/ folder of both wikis if needed for troubleshooting, please let me know a secure channel to provide them if needed since I can't provide it's contents pubicly.

[splitbrain]

Did the clustering step actually succeed for the second wiki? Have a look at the raw database and check if your chunks have clusters assigned.

TBH, at these sizes I would recommend going with a proper vector storage instead of sqlite.

[eduardomozart]

It doesn't seems to have chunks assigned (SELECT * FROM clusters; - the cluster tables are empty) and I don't know why. In the working Wiki, there's clusters assigned. I may try to clean and generate the SQLite database again in the non-working Wiki.

[splitbrain]

try running the maintenance command again. most likely it ran out of memory.