Fold testcase_run.sh logic into judgedaemon.

The primary motivation is to significantly reduce the per-testcase
overhead of judging a submission.

Historically, with fewer test cases, the separation was carrying its
weight. However, with modern problems requiring many test cases or
passes, the overhead from dozens of forked simple programs (`cp`, `mv`,
`chmod`, `grep` etc.) within the shell script has become a major
performance bottleneck. Moving them into PHP effectively replaces them
with simple system calls.

This change results in a substantial performance improvement:
* Roughly a 50% speed-up in total judging overhead per submission
* Example: A simple C++ solution to a multi-pass problem (NWERC 2025
  practice) was reduced from ~70s to ~33s end-to-end.

Author: meisterT

.github/workflows/judge-unit-tests.yml

@@ -0,0 +1,37 @@
+name: Judge unit tests
+on:
+  merge_group:
+  pull_request:
+    branches:
+      - main
+      - '[0-9]+.[0-9]+'
+
+jobs:
+  judge-unit-tests:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        php-version: ['8.2', '8.3', '8.4']
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Setup PHP
+        uses: shivammathur/setup-php@v2
+        with:
+          php-version: ${{ matrix.php-version }}
+
+      - name: Cache composer dependencies
+        uses: actions/cache@v4
+        with:
+          path: webapp/vendor
+          key: composer-${{ hashFiles('webapp/composer.lock') }}
+          restore-keys: |
+            composer-
+
+      - name: Install composer dependencies
+        working-directory: webapp
+        run: composer install --no-scripts --ignore-platform-reqs
+
+      - name: Run judge unit tests
+        working-directory: judge/tests
+        run: ../../webapp/vendor/bin/phpunit --configuration phpunit.xml --testdox

doc/manual/config-advanced.rst

@@ -223,18 +223,16 @@ To allow for problems that do not fit within the standard scheme of
 fixed input and/or output, DOMjudge has the possibility to change the
 way submissions are run and checked for correctness.
 
-The back end script ``testcase_run.sh`` that handles
-the running and checking of submissions, calls separate programs
-for running submissions and comparison of the results. These can be
-specialised and adapted to the requirements per problem. For this, one
+The judgedaemon that handles the running and checking of submissions, calls
+separate programs for running submissions and comparison of the results. These
+can be specialised and adapted to the requirements per problem. For this, one
 has to create executable archives as described above.
-Then the executable must be
-selected in the ``special_run`` and/or ``special_compare``
-fields of the problem (an empty value means that the default run and
-compare scripts should be used; the defaults can be set in the global
-configuration settings). When creating custom run and compare
-programs, we recommend reusing wrapper scripts that handle the
-tedious, standard part. See the boolfind example for details.
+Then the executable must be selected in the ``special_run`` and/or
+``special_compare`` fields of the problem (an empty value means that the
+default run and compare scripts should be used; the defaults can be set in the
+global configuration settings). When creating custom run and compare programs,
+we recommend reusing wrapper scripts that handle the tedious, standard part.
+See the boolfind example for details.
 
 Compare programs
 ----------------
@@ -257,8 +255,7 @@ output. The validator program should not make any assumptions on its
 working directory.
 
 For more details on writing and modifying a compare (or validator)
-script, see the ``boolfind_cmp`` example and the comments at the
-top of the file ``testcase_run.sh``.
+script, see the ``boolfind_cmp`` example.
 
 Run programs
 ------------

etc/judgehost-static.php.in

@@ -20,7 +20,7 @@ define('CHROOTDIR',   '@judgehost_chrootdir@');
 define('RUNUSER',     '@RUNUSER@');
 define('RUNGROUP',    '@RUNGROUP@');
 
-// Possible exitcodes from testcase_run.sh and their meaning.
+// Possible exitcodes from compile scripts and their meaning.
 $EXITCODES = array (
     0   => 'correct',
     101 => 'compiler-error',

example_problems/hello/submissions/no_output/test-timelimit-bug.c

@@ -3,7 +3,7 @@
  * This is issue #122 and fixed now, see old description below.
  *
  * The reason for TIMELIMIT was that program and runguard stderr are
- * mixed and searched by testcase_run.sh for the string 'timelimit exceeded'.
+ * mixed and searched by judgedaemon for the string 'timelimit exceeded'.
  * This a minor bug that doesn't provide a team any advantages. It
  * could be fixed by having runguard write the submission stderr to a
  * separate file.

example_problems/hello/submissions/time_limit_exceeded/stress-test-fork-setsid.c

@@ -1,15 +1,14 @@
 /*
  * When cgroups are enabled, this will hit timelimit and then be killed.
  *
- * Without cgroups however, this will crash the judging daemon: it
- * forks processes and places these in a new session, such that
- * testcase_run cannot retrace and kill these. They are left running
+ * Without cgroups however (not supported anymore), this will crash the judging
+ * daemon: it forks processes and places these in a new session, such that
+ * the judgedaemon cannot retrace and kill these. They are left running
  * and should be killed before restarting the judging daemon. The
  * cgroups code can detect this because the processes will belong to the
  * same cgroup.
  *
  * @EXPECTED_RESULTS@: TIMELIMIT
- * (or judgedaemon crash when cgroups disabled)
  */
 
 #include <unistd.h>

example_problems/hello/submissions/time_limit_exceeded/test-fork.c

@@ -4,7 +4,7 @@
  * timeout.
  *
  * The result should be a TIMELIMIT and the running forked programs
- * killed by testcase_run.
+ * killed by the judgedaemon.
  *
  * @EXPECTED_RESULTS@: TIMELIMIT
  */

judge/.gitignore

@@ -6,3 +6,4 @@
 /evict
 /create-cgroups.service
 /[email protected]
+/tests/.phpunit.result.cache

judge/Makefile

@@ -24,7 +24,7 @@ runpipe: runpipe.cc $(LIBOBJECTS)
 
 install-judgehost:
 	$(INSTALL_PROG) -t $(DESTDIR)$(judgehost_libjudgedir) \
-		compile.sh build_executable.sh testcase_run.sh chroot-startstop.sh \
+		compile.sh build_executable.sh chroot-startstop.sh \
 		check_diff.sh evict version_check.sh
 	$(INSTALL_DATA) -t $(DESTDIR)$(judgehost_libjudgedir) \
 		judgedaemon.main.php run-interactive.sh