FEAT: Build Search Index with a button

[physikerwelt]

The documentation says:

Build Search Index (Step 3/3)
Run the following maintenance scripts to build the search index:
php extensions/CirrusSearch/maintenance/UpdateSearchIndexConfig.php
php extensions/CirrusSearch/maintenance/ForceSearchIndex.php --skipParse
php extensions/CirrusSearch/maintenance/ForceSearchIndex.php --skipLinks --indexOnSkip

You will need to re-run these whenever you add new properties to the configuration.

Similar to SMW it would be nice if there would be a button to build the search index. Especially when the configuration can be done on the wiki it has (unfortunately) turned out to be a burden for admins to log into the server shell.

[JeroenDeDauw]

Building the search index can take a long time, so this is not trivial to implement well.

[physikerwelt]

Building the search index can take a long time, so this is not trivial to implement well.

I know from mathsearch, but I saw that this is possible for SMW so I was hoping that one could copy that mechanism.

[physikerwelt]

One approximation might be to run

php extensions/CirrusSearch/maintenance/ForceSearchIndex.php --skipParse && php extensions/CirrusSearch/maintenance/ForceSearchIndex.php --skipLinks --indexOnSkip

in an infinite loop in a dedicated docker container. Does this command destroy the existing index, or can the search be used as before while the script is running?

^^ This can be implemented in just a few minutes

The next best approximation would be to store the revision of the page and only run the script if the revision was changed.

^^ This would be some work, maybe 1-2 hours.

[JeroenDeDauw]

I looked into this, and it turns out the rebuild is incremental. So yes, running it in a loop should theoretically not cause disruption to the search experience.

[malberts]

ForceSearchIndex.php creates new jobs. I tested running that 3 times leads to a ~3x increase in runJobs execution time. And the actual jobs take longer to complete than the ForceSearchIndex.php run itself. So it likely isn't a good idea to run this in a loop, or at least the interval between ForceSearchIndex.php runs should exceed the time to actually finish all the jobs.

[physikerwelt]

@malberts thank you. That makes it more complicated. Maybe the simplest functional algorithm is:

$last_revion_of_settigs_page = 0
do:
  if ( getRevisionOfSettingsPage() > $last_revion_of_settigs_page ):
     $last_revion_of_settigs_page = getRevisionOfSettingsPage();
     cancelEsWriteJobs();
     startIndexing();

[JeroenDeDauw]

That seems reasonable logic when using the job queue. Though I think it is best to avoid using the job queue, if possible. If you can execute the logic directly in your script without it being deferred to jobs, you end up with a simpler system, and you can't interfere with things that need to be jobs. The point of the job queue is to defer work, but this simply isn't needed/desired for this reindexing use case.

[physikerwelt]

I don’t know how I could achieve that. It would be better if some mechanism could be built into the extension, as all users of the extension will face this problem.

[physikerwelt]

I realized that reindexing is triggered when the templates change. Maybe the same machnism can be used when the on wiki config changes?