Merge branch 'master' into AA/anotherPythonFix

Author: Alami-Amine

.gitignore

@@ -12,3 +12,6 @@ infra/base-images/base-builder/indexer/fuzzing_engine.a
 # IntelliJ IDEA
 /.idea
 **/*.iml
+
+# Chronos
+ccaches/

docs/faq.md

@@ -105,7 +105,30 @@ We currently do not have a good way to deduplicate timeout or OOM bugs.
 So, we report only one timeout and only one OOM bug per fuzz target.
 Once that bug is fixed, we will file another one, and so on.
 
-Currently we do not offer ways to change the memory and time limits.
+## Can I change the default timeout and OOM for a fuzz target?
+
+Yes, you can. For this, create a fuzz target options file named `<fuzz-target>.options`,
+where `<fuzz-target>` is the executable file name of the fuzz target, in the same
+directory as your `project.yaml`. The options file can contain fuzzer-specific
+configuration values, such as:
+
+```
+[libfuzzer]
+rss_limit_mb = 6000
+timeout = 30
+```
+
+## My library gracefully handles allocation failures, why are OOMs reported?
+
+OOM detection is done *not* by instrumenting memory allocation routines such as `malloc`
+to have them return NULL, but using a separate watchdog thread that measures the resident
+set size (RSS) on a periodic basis. Therefore, your fuzz target might successfully
+allocate more than the configured max RSS, and yet get killed shortly afterwards.
+
+The only reliable way to avoid this is for your fuzz target to use a custom allocator
+that will prevent allocating more memory than a given limit. You can find a more
+detailed discussion of this topic, as well as links to the solution implemented
+by a specific project, in [this issue](https://github.com/google/oss-fuzz/issues/1830).
 
 ## Can I launch an additional process (e.g. a daemon) from my fuzz target?
 

docs/further-reading/fuzzer_environment.md

@@ -25,13 +25,17 @@ or built as part of
 are not available on the bot runtime environment (where the fuzz targets run).
 
 If you need these dependencies in the runtime environment, you can either:
-- Install the packages via Dockerfile
+
+- (recommended) Build the dependencies statically in
+[build.sh]({{ site.baseurl }}/getting-started/new-project-guide/#buildsh)
+([example](https://github.com/google/oss-fuzz/blob/64f8b6593da141b97c98c7bc6f07df92c42ee010/projects/ffmpeg/build.sh#L26)).
+
+- Or install the packages via Dockerfile
 ([example](https://github.com/google/oss-fuzz/blob/2d5e2ef84f281e6ab789055aa735606d3122fda9/projects/tor/Dockerfile#L19))
 and then link statically against them
 ([example](https://github.com/google/oss-fuzz/blob/2d5e2ef84f281e6ab789055aa735606d3122fda9/projects/tor/build.sh#L40)).
-- Or build the dependencies statically in
-[build.sh]({{ site.baseurl }}/getting-started/new-project-guide/#buildsh)
-([example](https://github.com/google/oss-fuzz/blob/64f8b6593da141b97c98c7bc6f07df92c42ee010/projects/ffmpeg/build.sh#L26)).
+**Dependencies built in this way will not be instrumented** and may prevent
+the fuzzer from finding bugs if they are involved in the execution of a fuzz target.
 
 All build artifacts needed during fuzz target execution should be inside the
 `$OUT` directory. Only those artifacts are archived and used on the bots.

docs/getting-started/continuous_integration.md

@@ -234,6 +234,21 @@ You can checkout CIFuzz configs for OSS-Fuzz projects. Example -
 [systemd](https://github.com/systemd/systemd/blob/main/.github/workflows/cifuzz.yml),
 [curl](https://github.com/curl/curl/blob/master/.github/workflows/fuzz.yml).
 
+## Ubuntu 24.04 Support
+
+CIFuzz supports building and running fuzzers in an Ubuntu 24.04 environment.
+Existing projects will continue to use the legacy environment (Ubuntu 20.04) by default,
+preserving current behavior.
+
+To migrate your project to Ubuntu 24.04, add the following line to your `project.yaml`:
+
+```yaml
+base_os_version: ubuntu-24-04
+```
+
+For OSS-Fuzz projects, this file is located at `projects/<project_name>/project.yaml`.
+For external projects (ClusterFuzzLite), this file is typically located at `.clusterfuzzlite/project.yaml`.
+
 ## Understanding results
 
 The results of CIFuzz can be found in two different places.

docs/getting-started/new_project_guide.md

@@ -237,6 +237,10 @@ For an example, see
 [ecc-diff-fuzzer/Dockerfile](https://github.com/google/oss-fuzz/blob/master/projects/ecc-diff-fuzzer/Dockerfile).
 where we use `base-builder-rust`and install golang
 
+Runtime dependencies of your project, such as third-party static libraries, will
+not be instrumented if you build them in the Dockerfile. In most cases, you will
+want to build them in `build.sh` instead.
+
 ## build.sh {#buildsh}
 
 This file defines how to build binaries for [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) in your project.

infra/base-images/all.sh

@@ -62,7 +62,7 @@ for image_name in ${IMAGE_LIST}; do
   fi
 
   echo "Building ${tag} from ${dockerfile}..."
-  docker build --pull -t "${tag}" -f "${dockerfile}" "${image_dir}"
+  docker build -t "${tag}" -f "${dockerfile}" "${image_dir}"
 done
 
-echo "All builds for version ${VERSION_TAG} completed successfully."
+echo "All builds for version ${VERSION_TAG} completed successfully."

infra/base-images/base-builder/indexer/index_build.py

@@ -61,6 +61,13 @@
 # wrapper. Get around this by writing to a file instead.
 COMPILE_SETTINGS_PATH = INDEXER_DIR / 'compile_settings.json'
 
+CLANG_TOOLCHAIN_BINARY_PREFIXES = (
+    'clang-',
+    'ld',
+    'lld',
+    'llvm-',
+)
+
 EXTRA_CFLAGS = (
     '-fno-omit-frame-pointer '
     '-DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION '
@@ -125,11 +132,15 @@ def set_up_wrapper_dir():
     shutil.rmtree(_TOOLCHAIN_WITH_WRAPPER)
   _TOOLCHAIN_WITH_WRAPPER.mkdir(parents=True)
 
-  # Set up symlinks to every binary except for clang.
+  # Set up symlinks to toolchain binaries.
   wrapper_bin_dir = _TOOLCHAIN_WITH_WRAPPER / 'bin'
   wrapper_bin_dir.mkdir()
   for name in os.listdir(_CLANG_TOOLCHAIN / 'bin'):
-    if name in ('clang', 'clang++'):
+    # Symlink clang/llvm toolchain binaries, except for clang itself.
+    # We have to be careful not to symlink other unrelated binaries, since other
+    # parts of the build process may wrap those binaries (e.g.
+    # make_build_replayable.py in OSS-Fuzz).
+    if not name.startswith(CLANG_TOOLCHAIN_BINARY_PREFIXES):
       continue
 
     os.symlink(_CLANG_TOOLCHAIN / 'bin' / name, wrapper_bin_dir / name)

infra/base-images/base-builder/indexer/manifest_constants.py

@@ -0,0 +1,56 @@
+#!/usr/bin/env python3
+# Copyright 2025 Google LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Constants pertaining to snapshot manifests."""
+
+# pylint: disable=g-import-not-at-top
+try:
+  import pathlib
+
+  Path = pathlib.Path
+except ImportError:
+  import pathlib
+
+  Path = pathlib.Path
+
+
+# Source directory.
+SRC_DIR = Path("src")
+# Object directory.
+OBJ_DIR = Path("obj")
+# Directory for indexer data.
+INDEX_DIR = Path("idx")
+# Relative source file root in the index.
+INDEX_RELATIVE_SOURCES = INDEX_DIR / "relative"
+# Absolute source file root in the index.
+INDEX_ABSOLUTE_SOURCES = INDEX_DIR / "absolute"
+# The index database filename.
+INDEX_DB = Path("db.sqlite")
+# Library directory, where shared libraries are copied - inside obj.
+LIB_DIR = OBJ_DIR / "lib"
+# Manifest location
+MANIFEST_PATH = Path("manifest.json")
+
+
+# Where archive version 1 expects the lib directory to be mounted.
+LIB_MOUNT_PATH_V1 = Path("/ossfuzzlib")
+
+# Will be replaced with the input file for target execution.
+INPUT_FILE = "<input_file>"
+# A file the target can write output to.
+OUTPUT_FILE = "<output_file>"
+# Will be replaced with any dynamic arguments.
+DYNAMIC_ARGS = "<dynamic_args>"
+

infra/base-images/base-builder/indexer/manifest_types.py

@@ -36,23 +36,22 @@
 from typing import Any, Callable, Mapping, Self, Sequence
 import urllib.request
 
+import manifest_constants
 import pathlib
 
 
-# Source directory.
-SRC_DIR = pathlib.Path("src")
-# Object directory.
-OBJ_DIR = pathlib.Path("obj")
-# Directory for indexer data.
-INDEX_DIR = pathlib.Path("idx")
-# The index database filename.
-INDEX_DB = pathlib.Path("db.sqlite")
-# Library directory, where shared libraries are copied - inside obj.
-LIB_DIR = OBJ_DIR / "lib"
-# Manifest location
-MANIFEST_PATH = pathlib.Path("manifest.json")
-# Where archive version 1 expects the lib directory to be mounted.
-_LIB_MOUNT_PATH_V1 = pathlib.Path("/ossfuzzlib")
+SRC_DIR = manifest_constants.SRC_DIR
+OBJ_DIR = manifest_constants.OBJ_DIR
+INDEX_DIR = manifest_constants.INDEX_DIR
+INDEX_DB = manifest_constants.INDEX_DB
+LIB_DIR = manifest_constants.LIB_DIR
+MANIFEST_PATH = manifest_constants.MANIFEST_PATH
+LIB_MOUNT_PATH_V1 = manifest_constants.LIB_MOUNT_PATH_V1
+
+INPUT_FILE = manifest_constants.INPUT_FILE
+OUTPUT_FILE = manifest_constants.OUTPUT_FILE
+DYNAMIC_ARGS = manifest_constants.DYNAMIC_ARGS
+
 # Min archive version we currently support.
 _MIN_SUPPORTED_ARCHIVE_VERSION = 1
 # The current version of the build archive format.
@@ -64,15 +63,6 @@
                       f"latest_report_info/{os.getenv('PROJECT_NAME')}.json")
 
 
-# Will be replaced with the input file for target execution.
-INPUT_FILE = "<input_file>"
-# A file the target can write output to.
-OUTPUT_FILE = "<output_file>"
-# Will be replaced with any dynamic arguments.
-DYNAMIC_ARGS = "<dynamic_args>"
-
-
-
 class RepositoryType(enum.StrEnum):
   """The type of repository."""
 
@@ -142,12 +132,13 @@ class BinaryConfig:
 
   Attributes:
     kind: The kind of binary configuration.
-    binary_args: The arguments to pass to the binary, for example
-      "<input_file>".
+    binary_name: The name of the executable file.
   """
 
   kind: BinaryConfigKind
 
+  binary_name: str
+
   @classmethod
   def from_dict(cls, config_dict: Mapping[str, Any]) -> Self:
     """Deserializes the correct `BinaryConfig` subclass from a dict."""
@@ -186,7 +177,6 @@ class HarnessKind(enum.StrEnum):
 class CommandLineBinaryConfig(BinaryConfig):
   """Configuration for a command-line userspace binary."""
 
-  binary_name: str
   binary_args: list[str]
   # Additional environment variables to pass to the binary. They will overwrite
   # any existing environment variables with the same name.
@@ -285,7 +275,7 @@ class Manifest:
   def from_dict(cls, data: dict[str, Any]) -> Self:
     """Creates a Manifest object from a deserialized dict."""
     if data["version"] == 1:
-      lib_mount_path = _LIB_MOUNT_PATH_V1
+      lib_mount_path = LIB_MOUNT_PATH_V1
     else:
       lib_mount_path = _get_mapped(data, "lib_mount_path", pathlib.Path)
     if data["version"] < 3:
@@ -358,7 +348,7 @@ def validate(self) -> None:
           f"Build archive version too high: {self.version}. Only supporting"
           f" up to {ARCHIVE_VERSION}."
       )
-    if self.version == 1 and _LIB_MOUNT_PATH_V1 != self.lib_mount_path:
+    if self.version == 1 and LIB_MOUNT_PATH_V1 != self.lib_mount_path:
       raise RuntimeError(
           "Build archive with version 1 has an alternative lib_mount_path set"
           f" ({self.lib_mount_path}). This is not a valid archive."
@@ -409,12 +399,6 @@ def save_build(
 
     self.validate()
 
-    if not hasattr(self.binary_config, "binary_name"):
-      raise RuntimeError(
-          "Attempting to save a binary config type without binary_name."
-          " This is not yet supported. Kind: {self.binary_config.kind}."
-      )
-
     with tempfile.NamedTemporaryFile() as tmp:
       mode = "w:gz" if archive_path.suffix.endswith("gz") else "w"
       with tarfile.open(tmp.name, mode) as tar:
@@ -459,7 +443,9 @@ def _save_dir(
 
         dumped_self = self
         if self.index_db_version is None:
-          index_db_version = _get_sqlite_db_user_version(index_dir / INDEX_DB)
+          index_db_version = _get_sqlite_db_user_version(
+              pathlib.Path(index_dir) / INDEX_DB
+          )
           dumped_self = dataclasses.replace(
               self, index_db_version=index_db_version
           )