chronos: refactor and adjust documentation

infra/experimental/chronos/README.md

@@ -5,9 +5,29 @@ and analysis of projects' testing infrastructure. This is used by projects,
 e.g. [OSS-Fuzz-gen](https://github.com/google/oss-fuzz-gen) to help speed up
 valuation processes during fuzzing harness generation.
 
-Chronos is focused on two features, rebuilding projects fast and running the tests of a given project.
+At the core, Chronos relies on caching containers after project build, in order
+to enable fast rebuilding of a project following minor patches, and also enable
+running of the tests in a given project. To support this, Chronos creates a snapshot
+of a the docker container given project post build completion. This means that all `.o` files, generated
+configuations etc. persist in the docker container. These artifacts are then
+leverage for future "replay builds" where only a minor part of the project has changed,
+e.g. due to some patching on the project. This patching could be e.g. minor adjustments
+to the fuzzing harness source code e.g. by [oss-fuzz-gen](https://github.com/google/oss-fuzz-gen).
 
-## Rebuilding projects fast
+As such, at the core of Chronos are cached containers that are generated by taking a
+snapshot of the container of a project post project build.
+
+Chronos is focused on two features, rebuilding projects fast and running the
+tests of a given project.
+
+
+## Chronos features: fast rebuilding and running the tests of a project
+
+### CLI interface for Chronos
+
+The default route to validating Chronos is using the CLI available in `infra/experimental/chronos/manager.py`
+
+### Chronos feature: Fast rebuilding
 
 Chronos enables rebuilding projects efficiently in contexts where only a small patch
 needs to be evalualted in the target. This is achieved by running a replay build script
@@ -17,24 +37,28 @@ of the original `build.sh`.
 
 The replay build scripts are constructed in two ways: manually or automatically.
 
-### Automated rebuilds
+#### Option 1: Automated rebuilds
 
-Chronos support automated rebuilding by:
+Chronos support automated rebuilding. This is meant as a generic mechanism to enable Chronos support for projects by default. This is achieved by:
 
 1. Calling into a `replay_build.sh` script during the building inside the container [here](https://github.com/google/oss-fuzz/blob/206656447b213fb04901d15122692d8dd4d45312/infra/base-images/base-builder/compile#L292-L296)
 2. The `replay_build.sh` calls into `make_build_replayable.py`: [here](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-builder/replay_build.sh)
 3. `make_build_replayable.py` adjusts the build environment to wrap around common commands, to avoid performing a complete run of `build.sh`: [here](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-builder/make_build_replayable.py).
 
-### Manually provided replay builds
+The automated rebuilding works in combination with [Ccache](https://ccache.dev/), in order to facilitate cachine of e.g. `.o` files.
+This means that during rebuild mode as long as we have a cacche, we don't need to e.g. run `configure` again and will only have to
+rebuild the changed source code.
+
+#### Option 2: Manually provided replay builds
 
 `replay_build.sh` above, is simply just a wrapper script around `build.sh` that aims to enable
 fast rebuilding of the project. This `replay_build.sh` can, however, be overwritten in the Dockerfile
-of the project's builder image. Examples of this is [php](https://github.com/google/oss-fuzz/blob/206656447b213fb04901d15122692d8dd4d45312/projects/php/replay_build.sh#L1) and [ffmpeg](https://github.com/google/oss-fuzz/blob/master/projects/ffmpeg/replay_build.sh#L1).
+of the project's builder image to support a custom approach to fast rebuilding. Two examples of this is [php](https://github.com/google/oss-fuzz/blob/206656447b213fb04901d15122692d8dd4d45312/projects/php/replay_build.sh#L1) and [ffmpeg](https://github.com/google/oss-fuzz/blob/master/projects/ffmpeg/replay_build.sh#L1).
 
 Providing a manual `replay_build.sh` is likely more efficient at build time and can help speed up the process. Automated replay build scripts can also be erroneous.
 
 
-### Testing the validity of a replay build
+#### Testing the validity of a replay build
 
 The Chronos manager can use the `manager.py` to validate the validity of a
 replay build for a given project:
@@ -46,96 +70,51 @@ python3 infra/experimental/chronos/manager.py check-test tinyobjloader
 If the above command fails for the relevant project, then the replay build feature
 does not work for the given project.
 
-## Running tests of a project
+### Chronos feature:  Running tests of a project
 
 The second part of Chronos is a feature to enable running the tests of a given
 project. This is done by way of a script `run_tests.sh`. Samples of
 this script include [jsonnet](https://github.com/google/oss-fuzz/blob/master/projects/jsonnet/run_tests.sh#L1) and [tinyobjloader](https://github.com/google/oss-fuzz/blob/master/projects/tinyobjloader/run_tests.sh#L1).
 
 
-### Testing the validity of run_tests.sh
+#### Run tests constraints
+
+1. The `run_tests.sh` main task is to run the tests of a project and return `0` upon success and non-null otherwise.
+2. The `run_tests.sh` script must leave the main repository in the state as it was prior to the execution of `run_tests.sh` relative to `git diff` (or similar diff features of other version control systems).
+
+#### Testing the validity of run_tests.sh
 
 The Chronos manager can use the `manager.py` to validate the validity of a
 `run_tests.sh` script:
 
 ```sh
-python3 infra/experimental/chronos/manager.py
+python3 infra/experimental/chronos/manager.py check-tests json-c
 ```
 
 
-**Running tests of a project**
-
-## Pre-built images.
-
-Daily pre-built images are available at:
-
-- `us-central1-docker.pkg.dev/oss-fuzz/oss-fuzz-gen/<PROJECT>-ofg-cached-address`
-- `us-central1-docker.pkg.dev/oss-fuzz/oss-fuzz-gen/<PROJECT>-ofg-cached-coverage`
-
-They can be used as drop-in replacements for the usual `gcr.io/oss-fuzz/<PROJECT>` images.
-
-These images are generated in 2 ways:
-- (Preferred) [Generate](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-builder/bash_parser.py)
-  a replay build script that can be re-run alongside existing build artifacts,
-  leveraging existing build system mechanisms to avoid rebuilding (e.g. running
-  `make` twice should not actually rebuild everything). This is error-prone, so
-  we validate the script works by running it.
-- (Fallback, if the replay build script didn't work). We leverage
-  [ccache](https://ccache.dev/), to provide a compiler cache. This is often not
-  as fast as the replay build script, because some project builds spend
-  significant time doing non-compiler tasks (e.g. checking out submodules,
-  running configure scripts).
-
-Note: this mechanism does not work for every single OSS-Fuzz project today. The
-resulting image may either:
-- Not provide much performance improvement compared with a normal image, or
-- Not exist at all (if neither approach worked).
-
-Stats from a recent run: <https://gist.github.com/oliverchang/abaf3a1106a2b923c0ac3a577410aaaa>
-(Feb 3 2025).
-
-## Usage locally
-
-**Example 1: htslib**
+### Constraints imposed on replay_build.sh and run_tests.sh
 
-From the OSS-Fuzz root
+At the core of chronos are the two scripts `replay_build.sh` and `run_tests.sh`. We have a default
+mechanism for `replay_build.sh` so it's not strictly necessary to have a custom one, although it will
+likely improve speed and maybe correctness by providing one.
 
-```sh
-$ RUN_ALL=1 ./infra/experimental/chronos/build_cache_local.sh htslib c address
-...
-...
-Vanilla compile time:
-17
-Replay worked
-Replay compile time:
-2
-Ccache compile time: 
-9
-```
+There are three stages of the Chronos workflow:
 
+1. The cached containers represent the state of a build container after a successful project build.
+2. The `replay_build.sh` is able to rebuild a given project from the state of a cached container.
+3. The `run_tests.sh` script is able to run the tests of a given project. This should be able to succeed following the running of a `replay_build.sh`.
 
-## Check tests
+The stages (2) and (3) must both support running without network connectivity.
+Specifically, this means that the `replay_build.sh` must not do tasks e.g. fetch
+dependencies, download corpus, or anything of this nature. Similarly, the `run_tests.sh`
+must be able to operate completely in a closed network environment.
 
-Another feature of Chronos is the ability to run tests in a replayed build.
-This requires `run_tests.sh` to be available in the cached image at
-`$SRC/run_tests.sh`.
 
-Sample running:
+## Pre-built images.
 
-```
-$ git clone https://github.com/google/oss-fuzz
-$ cd oss-fuzz
-$ ./infra/experimental/chronos/check_tests.sh jsonnet
-...
-...
-100% tests passed, 0 tests failed out of 10
-
-Total Test time (real) = 119.80 sec
-```
+Chronos cached images are build daily, and pre-built images are available at:
 
-In order ot make the above work, the general approach is to have a
-`run_tests.sh` script in the OSS-Fuzz project's folder, which is copied into
-the main image.
+- `us-central1-docker.pkg.dev/oss-fuzz/oss-fuzz-gen/<PROJECT>-ofg-cached-address`
+- `us-central1-docker.pkg.dev/oss-fuzz/oss-fuzz-gen/<PROJECT>-ofg-cached-coverage`
 
-Notice that the `run_tests.sh` is run from a cached image, meaning the
-`run_tests.sh` is run after a run of building fuzzers.
+They can be used as drop-in replacements for the usual `gcr.io/oss-fuzz/<PROJECT>` images.