let the indexParallel() exceptions bubble up + removal of per directory reindex

[vladak]

This change contains couple of notable changes:

• more verbose Java/system/environment information reporting for both indexer and webapp
• the InterruptedException/ExecutionExecption from indexParallel() now bubble up to the indexer, altering the exit value of the indexer process so that automation can e.g. wipe out the index and reindex from scratch.
• removal of the capability to index individual directories (which is probably very rarely used if at all and potential source of index inconsistencies), simplifying IndexDatabase. Any extra indexer arguments are now treated as project paths.

In case of OOM error in one of the indexParallel tasks, the indexer logs look like this:

SEVERE: 6897 successes (100.0%) after aborting parallel-indexing
Jan 07, 2025 11:02:52 AM org.opengrok.indexer.index.IndexDatabase lambda$updateAll$5
SEVERE: Problem updating index database in directory '/var/opengrok/data.lucene/index/lucene': 
org.opengrok.indexer.index.IndexerException: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: fake OOM error
	at org.opengrok.indexer.index.IndexDatabase.indexParallel(IndexDatabase.java:1931)
	at org.opengrok.indexer.index.IndexDatabase.update(IndexDatabase.java:689)
	at org.opengrok.indexer.index.IndexDatabase.lambda$updateAll$5(IndexDatabase.java:352)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: fake OOM error
	at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
	at org.opengrok.indexer.index.IndexDatabase.indexParallel(IndexDatabase.java:1923)
	... 7 more
Caused by: java.lang.OutOfMemoryError: fake OOM error
	at org.opengrok.indexer.index.IndexDatabase.lambda$indexParallel$8(IndexDatabase.java:1913)
	... 4 more

Jan 07, 2025 11:02:52 AM org.opengrok.indexer.index.Indexer runMain
SEVERE: Indexer failed with IndexerException
Jan 07, 2025 11:02:52 AM org.opengrok.indexer.index.Indexer runMain
INFO: Suppressed exceptions (1 in total):
Jan 07, 2025 11:02:52 AM org.opengrok.indexer.index.Indexer runMain
INFO: 1: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: fake OOM error
Jan 07, 2025 11:02:52 AM org.opengrok.indexer.util.Statistics logIt
INFO: Indexer finished (took 30.67 seconds)

Process finished with exit code 1

[vladak]

To test this, I modified indexParallel() temporarily with:

diff --git a/opengrok-indexer/src/main/java/org/opengrok/indexer/index/IndexDatabase.java b/opengrok-indexer/src/main/java/org/opengrok/indexer/index/IndexDatabase.java
index 2c85ca2a340..20fc7d7384a 100644
--- a/opengrok-indexer/src/main/java/org/opengrok/indexer/index/IndexDatabase.java
+++ b/opengrok-indexer/src/main/java/org/opengrok/indexer/index/IndexDatabase.java
@@ -1908,6 +1908,11 @@ public class IndexDatabase {
                                 }
                             }
 
+                            // TODO: debug/test only, REMOVE BEFORE INTEGRATION !!!
+                            if (x.path.contains("a")) {
+                                throw new OutOfMemoryError("fake OOM error");
+                            }
+
                             progress.increment();
                             return x;
                         }