move fuzz/build.sh to oss-fuzz to fix fuzzing build error

[xingyaner]

The fix involves moving the fuzz/build.sh file to the oss-fuzz/projects/croaring/ directory and applying specific modifications to correct the fuzzing build failures. I will subsequently submit a Pull Request to the oss-fuzz repository to implement this solution.
https://github.com/RoaringBitmap/CRoaring/issues/762
https://oss-fuzz-build-logs.storage.googleapis.com/log-73d6d8bb-72ed-4d72-b403-5b1ef5afb33b.txt

[lemire]

Wouldn't it be easier to fix the fuzz/build.sh that is in CRoaring instead? That is, instead of deleting the file here, why don't you submit a fix to it here?

[lemire]

The CI failures are assuredly not related.

[xingyaner]

I regret to clarify that creating a dedicated build.sh within the OSS-Fuzz project structure is necessary because our project's build system employs a highly integrated configuration for compilers and flags. This design expects full autonomy over the build process and does not readily accommodate external control.

The fundamental issue is a binary format mismatch: the Fuzzer object files are fully instrumented with Sanitizers using OSS-Fuzz's toolchain, while the libroaring. a library is not. This incompatibility prevents them from being merged during the linking phase, resulting in the linker error.

The reason modifying CRoaring/fuzz/build.sh proves ineffective lies in the core design philosophy of the OSS-Fuzz build system, specifically its "convention over configuration" principle, which is strictly enforced by the infra/helper.py script. Its behavior is determined as follows:

1. If oss-fuzz/projects/croaring/build.sh exists
   The script enters a "custom build" mode. It ensures this specific build.sh becomes the content of /src/build.sh inside the container (our successful solution achieves this via the COPY instruction in the Dockerfile). The container's compile command then executes this authoritative script.

2. If oss-fuzz/projects/croaring/build.sh does not exist
   The script falls back to a "default build" mode. In this mode, it completely ignores any build scripts within the project's source code (such as CRoaring/fuzz/build.sh). It dynamically generates a generic, template-based build.sh and uses it to overwrite /src/build.sh upon container startup.

If you have an alternative approach, I would be glad to discuss it.

[lemire]

You have not answered my question in a clear manner.

We do not merge pull requests without a full understanding of what is involved.

Please use straight and direct language. You sound like an AI bot right now.

Your behaviour is suspicious.

I am closing this issue.

[xingyaner]

What I mean is that if build.sh only exists in croaring, the infra/helper.py script of oss-fuzz will ignore build.sh and the changes made to it. Instead, it will generate a template-based build.sh to override /src/build.sh in the container.

[lemire]

CRoaring has been tested by oss-fuzz for years, in the current form. It worked until a few weeks ago (up til october) and then the build failed.

https://oss-fuzz-build-logs.storage.googleapis.com/index.html#croaring

I maintain several other libraries that are fuzzed by oss-fuzz and the build.sh script is inside the repo, and they build fine.

We will happily build on your expertise, but please explain in clear terms:

1. Why did it start failing at the start of November ?
2. Why do countless projects have the script in the repo and work, while CRoaring suddenly could not.

We need to understand.

[xingyaner]

#758
Thank you for your reply.Of course, I think it's necessary to explain clearly. The root cause of the problem is as follows:
In Commit 8711167, a new line of code "set(CMAKE_INTERPROCEDURAL_OPTIMIZATION TRUE)" was added to CRoaring 's ‎CMakeLists.txt.This causes the format of the libroaring.a file generated by CMake calling the Clang compiler to change from ELF to LLVM Bitcode.

1. In the October 28th build, since libroaring.a is in ELF format, the OSS-Fuzz linker /usr/bin/ld can read and process it, so the fuzzing build is displayed as successful .

2. In the October 29th build, the linker failed to understand the Bitcode format, resulting in a failed build.

Error log shows "/usr/bin/ld: /src/croaring/build-dir/src/libroaring.a: error adding symbols: file format not recognized
clang++: error: linker command failed with exit code 1"

If we delete this line of code, it will terminate the IPO/LTO and restore the fuzzing build to the "successful" state. This is the simplest method.

[lemire]

Thanks! The issue should be fixed.

[xingyaner]

I'm very glad that my suggestion can be of some help.