chronos: disable network in replay builds and extend docs (google#14136)

The documentation needs a larger revamp, will do in a follow-up.

---------

Signed-off-by: David Korczynski <[email protected]>

Author: DavidKorczynski

infra/experimental/chronos/README.md

@@ -1,5 +1,70 @@
 # Chronos: rebuilding OSS-Fuzz harnesses using cached builds
 
+Chronos is a utility tooling to enable fast re-building of OSS-Fuzz projects
+and analysis of projects' testing infrastructure. This is used by projects,
+e.g. [OSS-Fuzz-gen](https://github.com/google/oss-fuzz-gen) to help speed up
+valuation processes during fuzzing harness generation.
+
+Chronos is focused on two features, rebuilding projects fast and running the tests of a given project.
+
+## Rebuilding projects fast
+
+Chronos enables rebuilding projects efficiently in contexts where only a small patch
+needs to be evalualted in the target. This is achieved by running a replay build script
+in the build container, similarly to how a regular `build_fuzzers` command would run, but
+with the caveat that the replay build script only performs a subset of the operations
+of the original `build.sh`.
+
+The replay build scripts are constructed in two ways: manually or automatically.
+
+### Automated rebuilds
+
+Chronos support automated rebuilding by:
+
+1. Calling into a `replay_build.sh` script during the building inside the container [here](https://github.com/google/oss-fuzz/blob/206656447b213fb04901d15122692d8dd4d45312/infra/base-images/base-builder/compile#L292-L296)
+2. The `replay_build.sh` calls into `make_build_replayable.py`: [here](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-builder/replay_build.sh)
+3. `make_build_replayable.py` adjusts the build environment to wrap around common commands, to avoid performing a complete run of `build.sh`: [here](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-builder/make_build_replayable.py).
+
+### Manually provided replay builds
+
+`replay_build.sh` above, is simply just a wrapper script around `build.sh` that aims to enable
+fast rebuilding of the project. This `replay_build.sh` can, however, be overwritten in the Dockerfile
+of the project's builder image. Examples of this is [php](https://github.com/google/oss-fuzz/blob/206656447b213fb04901d15122692d8dd4d45312/projects/php/replay_build.sh#L1) and [ffmpeg](https://github.com/google/oss-fuzz/blob/master/projects/ffmpeg/replay_build.sh#L1).
+
+Providing a manual `replay_build.sh` is likely more efficient at build time and can help speed up the process. Automated replay build scripts can also be erroneous.
+
+
+### Testing the validity of a replay build
+
+The Chronos manager can use the `manager.py` to validate the validity of a
+replay build for a given project:
+
+```sh
+python3 infra/experimental/chronos/manager.py check-test tinyobjloader
+```
+
+If the above command fails for the relevant project, then the replay build feature
+does not work for the given project.
+
+## Running tests of a project
+
+The second part of Chronos is a feature to enable running the tests of a given
+project. This is done by way of a script `run_tests.sh`. Samples of
+this script include [jsonnet](https://github.com/google/oss-fuzz/blob/master/projects/jsonnet/run_tests.sh#L1) and [tinyobjloader](https://github.com/google/oss-fuzz/blob/master/projects/tinyobjloader/run_tests.sh#L1).
+
+
+### Testing the validity of run_tests.sh
+
+The Chronos manager can use the `manager.py` to validate the validity of a
+`run_tests.sh` script:
+
+```sh
+python3 infra/experimental/chronos/manager.py
+```
+
+
+**Running tests of a project**
+
 ## Pre-built images.
 
 Daily pre-built images are available at:

infra/experimental/chronos/bad_patch.py

@@ -19,7 +19,7 @@
 import sys
 
 import tree_sitter_cpp
-from tree_sitter import Language, Node, Parser, Query, QueryCursor
+from tree_sitter import Language, Parser, Query, QueryCursor
 
 LANGUAGE = Language(tree_sitter_cpp.language())
 PARSER = Parser(LANGUAGE)
@@ -30,7 +30,27 @@
 
 def normal_compile():
   """Do nothing and act as a control test that should always success."""
-  pass
+
+
+def source_code_white_noise():
+  """Insert white noise. This is a control test which forces
+  recompilation of good code. We need this to make sure that
+  the system is able to rebuild full source code under the
+  Chronos environment."""
+  exts = ['.c', '.cc', '.cpp', '.cxx', '.h', '.hpp']
+  payload = '\n\n\n\n\n\n'
+
+  # Walk and insert garbage code
+  for cur, dirs, files in os.walk(ROOT_PATH):
+    dirs[:] = [d for d in dirs if d not in EXCLUDE_DIRS]
+    for file in files:
+      if any(file.endswith(ext) for ext in exts):
+        path = os.path.join(cur, file)
+        try:
+          with open(path, 'a', encoding='utf-8') as f:
+            f.write(payload)
+        except Exception:
+          pass
 
 
 def source_code_compile_error():
@@ -45,7 +65,7 @@ def source_code_compile_error():
       if any(file.endswith(ext) for ext in exts):
         path = os.path.join(cur, file)
         try:
-          with open(path, 'a') as f:
+          with open(path, 'a', encoding='utf-8') as f:
             f.write(payload)
         except Exception:
           pass
@@ -63,7 +83,7 @@ def macro_compile_error():
       if any(file.endswith(ext) for ext in exts):
         path = os.path.join(cur, file)
         try:
-          with open(path, 'a') as f:
+          with open(path, 'a', encoding='utf-8') as f:
             f.write(payload)
         except Exception:
           pass
@@ -92,13 +112,13 @@ def missing_header_error():
         try:
           # Read source file
           source = ''
-          with open(path, 'r') as f:
+          with open(path, 'r', encoding='utf-8') as f:
             source = f.read()
           if not source:
             continue
 
           # Append a wrong header inclusion at the beginning
-          with open(path, 'w') as f:
+          with open(path, 'w', encoding='utf-8') as f:
             f.write(payload)
             f.write(source)
           count += 1
@@ -126,11 +146,11 @@ def duplicate_symbol_error():
         try:
           # Try read and parse the source with tree-sitter
           source = ''
-          with open(path, 'r') as f:
+          with open(path, 'r', encoding='utf-8') as f:
             source = f.read()
           if source:
             node = PARSER.parse(source.encode()).root_node
-        except:
+        except Exception:
           pass
 
         if not node:
@@ -147,10 +167,10 @@ def duplicate_symbol_error():
         # Add source code with duplicated declaration randomly
         if new_source and random.choice([True, False]):
           try:
-            with open(path, 'w') as f:
+            with open(path, 'w', encoding='utf-8') as f:
               f.write(new_source)
             count += 1
-          except:
+          except Exception:
             pass
 
 
@@ -174,18 +194,18 @@ def function_linker_error():
         try:
           # Read source file
           source = ''
-          with open(path, 'r') as f:
+          with open(path, 'r', encoding='utf-8') as f:
             source = f.read()
           if not source:
             continue
 
           # Append a wrong header inclusion at the beginning
-          with open(path, 'w') as f:
+          with open(path, 'w', encoding='utf-8') as f:
             f.write(payload)
             f.write(source)
             f.write(payload_call.replace('{COUNT}', str(count)))
           count += 1
-        except:
+        except Exception:
           pass
 
 
@@ -194,6 +214,10 @@ def function_linker_error():
         'func': normal_compile,
         'rc': [0],
     },
+    'white_noise': {
+        'func': source_code_white_noise,
+        'rc': [0],
+    },
     'compile_error': {
         'func': source_code_compile_error,
         'rc': [1, 2],
@@ -218,6 +242,7 @@ def function_linker_error():
 
 
 def main():
+  """Main entrypoint."""
   target = sys.argv[1]
   BAD_PATCH_GENERATOR[target]['func']()
 

infra/experimental/chronos/manager.py

@@ -85,7 +85,7 @@ def build_project_image(project):
 def build_cached_project(project, cleanup=True, sanitizer='address'):
   """Build cached image for a project."""
   container_name = _get_project_cached_named_local(project, sanitizer)
-
+  logger.info('Building cached image for project: %s', project)
   # Clean up the container if it exists.
   if cleanup:
     try:
@@ -104,11 +104,16 @@ def build_cached_project(project, cleanup=True, sanitizer='address'):
       f'--env=FUZZING_LANGUAGE={project_language}',
       '--env=CAPTURE_REPLAY_SCRIPT=1', f'--name={container_name}',
       f'-v={cwd}/ccaches/{project}/ccache:/workspace/ccache',
-      f'-v={cwd}/build/out/{project}/:/out/', f'gcr.io/oss-fuzz/{project}',
-      'bash', '-c',
-      '"export PATH=/ccache/bin:\$PATH && compile && cp -n /usr/local/bin/replay_build.sh \$SRC/"'
+      f'-v={cwd}/build/out/{project}/:/out/',
+      '-v=' + os.path.join(os.getcwd(), 'infra', 'experimental', 'chronos') +
+      ':/chronos/', f'gcr.io/oss-fuzz/{project}', 'bash', '-c',
+      ('"export PATH=/ccache/bin:\$PATH && python3.11 -m pip install -r /chronos/requirements.txt && '
+       'rm -rf /out/* && compile && cp -n /usr/local/bin/replay_build.sh \$SRC/"'
+      )
   ]
 
+  logger.info('Command: %s', ' '.join(cmd))
+
   start = time.time()
   try:
     subprocess.check_call(' '.join(cmd), shell=True)
@@ -162,14 +167,18 @@ def build_cached_project(project, cleanup=True, sanitizer='address'):
 def check_cached_replay(project, sanitizer='address', integrity_test=False):
   """Checks if a cache build succeeds and times is."""
   build_project_image(project)
-  build_cached_project(project, sanitizer=sanitizer)
+  if not build_cached_project(project, sanitizer=sanitizer):
+    logger.info('Failed to build cached image for project: %s', project)
+    return
 
   start = time.time()
-  base_cmd = 'export PATH=/ccache/bin:$PATH && rm -rf /out/* && compile'
+  base_cmd = 'export PATH=/ccache/bin:\\$PATH && rm -rf /out/* && compile'
   cmd = [
       'docker',
       'run',
       '--rm',
+      '--network',
+      'none',
       '--env=SANITIZER=' + sanitizer,
       '--env=FUZZING_LANGUAGE=c++',
       '-v=' + os.path.join(os.getcwd(), 'build', 'out', project) + ':/out',
@@ -187,9 +196,7 @@ def check_cached_replay(project, sanitizer='address', integrity_test=False):
     for bad_patch_name, bad_patch_map in bad_patch.BAD_PATCH_GENERATOR.items():
       # Generate bad patch command using different approaches
       expected_rc = bad_patch_map['rc']
-      bad_patch_command = (
-          'python3 -m pip install -r /chronos/requirements.txt && '
-          f'python3 /chronos/bad_patch.py {bad_patch_name}')
+      bad_patch_command = (f'python3 /chronos/bad_patch.py {bad_patch_name}')
       cmd_to_run = cmd[:]
       cmd_to_run.append(
           f'"set -euo pipefail && {bad_patch_command} && {base_cmd}"')
@@ -203,17 +210,25 @@ def check_cached_replay(project, sanitizer='address', integrity_test=False):
                      'Return code: %d. Expected return code: %s'), project,
                     bad_patch_name, result.returncode, str(expected_rc))
 
-      if failed:
-        logger.info(
-            '%s check cached replay failed to detect these bad patches: %s',
-            project, ' '.join(failed))
-      else:
-        logger.info('%s check cached replay success to detect all bad patches.',
-                    project)
+    if failed:
+      logger.info(
+          '%s check cached replay failed to detect these bad patches: %s',
+          project, ' '.join(failed))
+    else:
+      logger.info('%s check cached replay success to detect all bad patches.',
+                  project)
   else:
     # Normal run with no integrity check
     cmd.append(f'"{base_cmd}"')
-    subprocess.run(' '.join(cmd), shell=True, check=False)
+    replay_success = False
+    try:
+      subprocess.run(' '.join(cmd), shell=True, check=True)
+      replay_success = True
+    except subprocess.CalledProcessError as e:
+      logger.error('Failed to run cached replay: %s', e)
+      replay_success = False
+    logger.info('%s check cached replay: %s.', project,
+                'succeeded' if replay_success else 'failed')
 
   end = time.time()
   logger.info('%s check cached replay completion time: %.2f seconds', project,
@@ -266,7 +281,7 @@ def check_test(project,
     for logic_patch in logic_error_patch.LOGIC_ERROR_PATCHES:
       logger.info('Checking logic patch: %s', logic_patch.name)
       patch_command = (
-          'python3 -m pip install -r /chronos/requirements.txt && '
+          'python3 -m pip install -r /chronos/requirements.txt &&'
           f'python3 /chronos/logic_error_patch.py {logic_patch.name} && '
           'compile')
       cmd_to_run = docker_cmd[:]
@@ -537,11 +552,9 @@ def _cmd_dispatcher_check_test(args):
 
 
 def _cmd_dispatcher_check_replay(args):
-  check_cached_replay(args.project, args.sanitizer)
-
-
-def _cmd_dispatcher_check_replay_integrity(args):
-  check_cached_replay(args.project, args.sanitizer, integrity_test=True)
+  check_cached_replay(args.project,
+                      args.sanitizer,
+                      integrity_test=args.integrity_test)
 
 
 def _cmd_dispatcher_build_cached_image(args):
@@ -616,20 +629,10 @@ def parse_args():
       '--sanitizer',
       default='address',
       help='The sanitizer to use for the cached build (default: address).')
-
-  check_replay_script_integrity_parser = subparsers.add_parser(
-      'check-replay-script-integrity',
-      help=
-      ('Checks if the replay script works for a specific project. '
-       'Integrity of the replay script is also tested with different bad patches.'
-      ))
-
-  check_replay_script_integrity_parser.add_argument(
-      'project', help='The name of the project to check.')
-  check_replay_script_integrity_parser.add_argument(
-      '--sanitizer',
-      default='address',
-      help='The sanitizer to use for the cached build (default: address).')
+  check_replay_script_parser.add_argument(
+      '--integrity-test',
+      action='store_true',
+      help='If set, will test the integrity of the replay script.')
 
   check_run_tests_script_parser = subparsers.add_parser(
       'check-run-tests-script',
@@ -723,7 +726,6 @@ def main():
   dispatch_map = {
       'check-test': _cmd_dispatcher_check_test,
       'check-replay-script': _cmd_dispatcher_check_replay,
-      'check-replay-script-integrity': _cmd_dispatcher_check_replay_integrity,
       'build-cached-image': _cmd_dispatcher_build_cached_image,
       'autogen-tests': _cmd_dispatcher_autogen_tests,
       'build-many-caches': _cmd_dispatcher_build_many_caches,