Export fmt dependency in CMake config

[kmilos]

No description provided.

[neheb]

has this been tested?

I wouldn't expect it to be needed.

[kmilos]

has this been tested?

Not yet, hence the draft still...

I wouldn't expect it to be needed.

It is no different to other dependencies for static linkage?

I.e. looks like we've already also missed it in the private link interface for the still failing fuzz test... @kevinbackhouse

[kevinbackhouse]

Hmm. I was hoping that the CIFuzz would start working by itself after #2769 was merged. I can see from the build log that it is installing the libfmt-dev package now, so I'm not sure why it's not working. The build script that OSS-Fuzz uses is a fairly basic use of our cmake system: https://github.com/google/oss-fuzz/blob/master/projects/exiv2/build.sh. Can either of you see what might be wrong with it?

[kmilos]

Not it. Looks like Ubuntu doesn't ship static libfmt.a in libfmt-dev...?

Edit: apart from MSYS2, looks like no distro is shipping the static lib (checked only major ones like Arch, Debian, Fedora)? In that case, this PR makes less sense indeed...

[kevinbackhouse]

Not it. Looks like Ubuntu doesn't ship static libfmt.a in libfmt-dev...?

Oh. So you think it's because of the -DBUILD_SHARED_LIBS=OFF? But we have other tests that do that.

[kevinbackhouse]

OSS-Fuzz is using Ubuntu 20.04, so maybe that's part of the issue.

[kmilos]

So you think it's because of the -DBUILD_SHARED_LIBS=OFF? But we have other tests that do that.

True that...

OSS-Fuzz is using Ubuntu 20.04, so maybe that's part of the issue.

Very likely, it ships an ancient version 6.1.2...

[kevinbackhouse]

Is there a cmake flag that will tell OSS-Fuzz to build without fmt? That might be the simplest solution. It's already building without the libinih library due to a similar issue.

[kmilos]

Not optional - it's either libfmt (we didn't check the minimum version? @neheb) or C++20 features in more recent libstdc++/libc++, but not without either option.

Might be time to bump requirements on main...

[kevinbackhouse]

Is there a flag to build with C++20? I could try that: I expect they offer that latest compiler versions on OSS-Fuzz even though they're on an old version of Ubuntu.

[kmilos]

Is there a flag to build with C++20? I could try that: I expect they offer that latest compiler versions on OSS-Fuzz even though they're on an old version of Ubuntu.

Looks like we might be in luck, it is indeed Clang 18... You can use the same flag we use for Cygwin builds (no libfmt there):

exiv2/.github/workflows/on_PR_windows_matrix.yml

Line 183 in f07ddc6

-DCMAKE_CXX_STANDARD=20 \

[neheb]

Is libfmt 6 not working?

[kmilos]

Apparently not linking for some reason w/ -DBUILD_SHARED_LIBS=OFF. I don't think that config is covered anywhere else in CI w/ that particular libfmt version...

[kevinbackhouse]

The C++20 idea didn't work: kevinbackhouse/oss-fuzz#12

But I just tried building in a ubuntu:focal docker container and it works fine (with or without the C++20 flag). I'm building with clang-18. So it must be something else about the OSS-Fuzz environment.

[neheb]

Apparently not linking for some reason w/ -DBUILD_SHARED_LIBS=OFF. I don't think that config is covered anywhere else in CI w/ that particular libfmt version...

tested ubuntu 20.04 with libfmt and clang++-10. works fine. exiv2 is not using special functionality in fmt. Just the std::format compatible stuff.

[neheb]

The C++20 idea didn't work: kevinbackhouse/oss-fuzz#12

But I just tried building in a ubuntu:focal docker container and it works fine (with or without the C++20 flag). I'm building with clang-18. So it must be something else about the OSS-Fuzz environment.

/src/exiv2/src/pgfimage.cpp:32:23: error: no member named 'byteswap' in namespace 'std'

That...is a bug that needs fixing.

[neheb]

@kevinbackhouse does this addition in src/pgfimage.cpp work?

#ifdef __cpp_lib_endian
#include <bit>
#endif

[kevinbackhouse]

@neheb: that std:byteswap error is a new thing that happened when I tried C++-23 rather than C++-20. I can't reproduce it on my own computer. It only happens in the OSS-Fuzz environment.

[neheb]

I'll test in #2645 to see if CI can reproduce. It's most likely a missing bit header.

[kevinbackhouse]

@kevinbackhouse does this addition in src/pgfimage.cpp work?

#ifdef __cpp_lib_endian
#include <bit>
#endif

Yes, this solves the std::byteswap error. Unfortunately I'm now getting the same linker error with C++-23.

I guess it might be worth making this change anyway though?

[neheb]

OK soo:

Ubuntu-20.04 does not come with the format header, so the check fails and fmt gets used. It uses the libc++ from gcc9. 13 introduces format.

libfmt gets found and used. But LD errors with

/usr/bin/ld: /usr/bin/ld: DWARF error: invalid or unhandled FORM value: 0x25

I think I've seen this but don't remember where.

Since oss-fuzz uses clang-18, would it be possible to also use lld-18?

edit: oooohhh that's exactly how I saw it. clang compiler with gcc's libc++ when I was testing meson CI.

[kmilos]

that's exactly how I saw it. clang compiler with gcc's libc++ when I was testing meson CI

There you install explicitly LLVM's libc++:

exiv2/.github/workflows/on_PR_meson.yaml

Line 44 in 8545614

sudo apt install -y libc++abi-${{matrix.cxx}}-dev libc++-${{matrix.cxx}}-dev

Maybe that's the missing piece?

[neheb]

No, that's unrelated.

[kmilos]

Also found this chestnut:

The implementation was complete since LLVM 14, but the feature-test macro was not set until LLVM 19

[neheb]

Unfortunate. Looks like the check needs to be adjusted.

[neheb]

anyway, CIFuzz failure. Some research led me to:

google/oss-fuzz#11698
https://gitlab.com/gnutls/gnutls/-/issues/1552

theory: libfmt is static, while everything else is shared, causing the error.
possible solution: build only static.

But then again I don't think that means cmake will use static versions of its dependencies.

I wonder if the issue is double linking.

Anyway, lld will definitely work.

[kevinbackhouse]

I tried modifying the build to use lld, but that doesn't work either: https://github.com/kevinbackhouse/oss-fuzz/actions/runs/12718238794/job/35456292416?pr=12

Do you think it's worth trying to build with meson rather than cmake, since you've already got that working on Ubuntu 20.04? Is there a way to build the fuzz build target with meson?

[neheb]

I tried modifying the build to use lld, but that doesn't work either: https://github.com/kevinbackhouse/oss-fuzz/actions/runs/12718238794/job/35456292416?pr=12

Do you think it's worth trying to build with meson rather than cmake, since you've already got that working on Ubuntu 20.04? Is there a way to build the fuzz build target with meson?

That error looks like missing -lfmt in the linker flags for exiv2_int

Tests are generally speaking unimplemented in meson. I have a WIP commit that doesn’t fully work. Problem is the tests require making modifications to the source directory, not the build one from what I remember.

Fuzz target also needs implementing.

[kmilos]

That error looks like missing -lfmt in the linker flags for exiv2_int

One would've thought this should do it:

exiv2/src/CMakeLists.txt

Line 256 in 99c307f

target_link_libraries(exiv2lib_int PRIVATE fmt::fmt)

[neheb]

CMAKE_VERBOSE_MAKEFILE would confirm that.

if I had to guess it’s some fmt cmake bug.

It would be nice to convert fully to meson since it works properly, but extra code is needed to get the tests and fuzzing working.

edit: I wonder if that lld on that run is lld-18 or older.

[kevinbackhouse]

I added VERBOSE=1 to the make command line: https://github.com/kevinbackhouse/oss-fuzz/actions/runs/12728906903/job/35480066657?pr=12

[kevinbackhouse]

I've started a discussion on the OSS-Fuzz repo to ask for help: google/oss-fuzz#12934
Other than that, the only thing I can think of is to create a minimized version of the problem. It'll be a slow process: I'll need to create a fork of Exiv2 and then keep removing files and directories from it until I've got a much smaller project that triggers the same error.

[kevinbackhouse]

Meanwhile, we'll have to ignore the CIFuzz failures. Would you like to merge this PR?

[kmilos]

Would you like to merge this PR?

I still think so...

[neheb]

OK I just accidentally hit this in a different project. My error was having

-stdlib=libc++

in CXXFLAGS.

edit: If OSS-Fuzz mandates llvm libc++, then fmt must absolutely be built as a subproject.

[kmilos]

Ah, so that's maybe why I've seen a few projects vendor libmft...

In any case, I guess the options are:

1. While waiting for OSS-Fuzz to hopefully bump to Clang 19, we fudge the detection a bit so c++20 std::format can be used
2. Wait for OSS-Fuzz to upgrade to Ubuntu 22.04 where libfmt.so is shipped
3. Build our own libmt for OSS-Fuzz