Invalid cache assertion building freetype port

[HCLJason]

I'm trying out an updated freetype port using cmake. When I run embuilder build freetype, I get a cache lock error on the first assert (line 39 in cache.py).

I added print(dict(os.environ)), right before it, and EM_CACHE_IS_LOCKED did not show up in it.

After commenting out the assert, it seems to hang when building freetype.

Please include the following in your bug report:

Version of emscripten/emsdk:
Please include the output emcc -v here

3.1.23

Failing command line in full:
If this is compile or link-time failure please include the full failing command
along with its entire output.

python3 ./embuilder.py build zlib freetype

embuilder:INFO: building freetype
ports:INFO: retrieving port: freetype from https://download.savannah.gnu.org/releases/freetype/freetype-2.12.1.tar.gz
ports:INFO: unpacking port: freetype
cache:INFO: generating port: sysroot/lib/wasm32-emscripten/libfreetype.a... (this will be cached in "/local/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libfreetype.a" for subsequent builds)
configure: cmake -B/local/emsdk/upstream/emscripten/cache/ports-builds/freetype -S/local/emsdk/upstream/emscripten/cache/ports/freetype/freetype-2.12.1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/local/emsdk/upstream/emscripten/cache/ports-builds/freetype -DCMAKE_CXX_FLAGS="-I/local/emsdk/upstream/emscripten/system/lib/libc/musl/include/" -DCMAKE_C_FLAGS="-I/local/emsdk/upstream/emscripten/system/lib/libc/musl/include/" -DCMAKE_TOOLCHAIN_FILE=/local/emsdk/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake -DCMAKE_CROSSCOMPILING_EMULATOR=/local/emsdk/node/14.18.2_64bit/bin/node;--experimental-wasm-threads
-- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)
CMake Warning at builds/cmake/FindHarfBuzz.cmake:99 (message):
Required version (2.0.0) is higher than found version ()
Call Stack (most recent call first):
CMakeLists.txt:253 (find_package)

-- Found ZLIB: /local/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libz.a (found version "1.2.12")
-- Could NOT find PNG (missing: PNG_LIBRARY PNG_PNG_INCLUDE_DIR)
-- Could NOT find BZip2 (missing: BZIP2_LIBRARIES BZIP2_INCLUDE_DIR)
-- Could NOT find BrotliDec (missing: BROTLIDEC_INCLUDE_DIRS BROTLIDEC_LIBRARIES)
-- Looking for unistd.h
-- Looking for unistd.h - not found

Traceback (most recent call last):
File "/local/emsdk/upstream/emscripten/emcc.py", line 4145, in
sys.exit(main(sys.argv))
File "/usr/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/local/emsdk/upstream/emscripten/emcc.py", line 4138, in main
ret = run(args)
File "/local/emsdk/upstream/emscripten/emcc.py", line 1192, in run
phase_calculate_system_libraries(state, linker_arguments, linker_inputs, newargs)
File "/usr/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/local/emsdk/upstream/emscripten/emcc.py", line 2860, in phase_calculate_system_libraries
extra_files_to_link += system_libs.calculate(all_linker_inputs, newargs, forced=state.forced_stdlibs)
File "/local/emscripten/tools/system_libs.py", line 2132, in calculate
libs_to_link = get_libs_to_link(args, forced, only_forced)
File "/local/emscripten/tools/system_libs.py", line 2068, in get_libs_to_link
add_library('libGL')
File "/local/emscripten/tools/system_libs.py", line 2014, in add_library
libs_to_link.append((lib.get_link_flag(), need_whole_archive))
File "/local/emscripten/tools/system_libs.py", line 379, in get_link_flag
fullpath = self.build()
File "/local/emscripten/tools/system_libs.py", line 371, in build
return shared.Cache.get(self.get_path(), self.do_build, force=USE_NINJA == 2, quiet=USE_NINJA)
File "/local/emscripten/tools/cache.py", line 145, in get
with self.lock(shortname):
File "/usr/lib/python3.8/contextlib.py", line 113, in enter
return next(self.gen)
File "/local/emscripten/tools/cache.py", line 65, in lock
self.acquire_cache_lock(reason)
File "/local/emscripten/tools/cache.py", line 42, in acquire_cache_lock
assert 'EM_CACHE_IS_LOCKED' not in os.environ, f'attempt to lock the cache while a parent process is holding the lock ({reason})'
AssertionError: attempt to lock the cache while a parent process is holding the lock (sysroot/lib/wasm32-emscripten/libGL.a)
make[1]: *** [CMakeFiles/cmTC_d8893.dir/build.make:88: cmTC_d8893.js] Error 1
make[1]: Leaving directory '/local/emscripten/cache/ports-builds/freetype/CMakeFiles/CMakeTmp'
make: *** [Makefile:121: cmTC_d8893/fast] Error 2

[sbc100]

We try to avoid using any external tools when building ports, since that would mean that any users of the port then need to have that tool installed. So we don't use make or ninja or cmake in tools/ports anywhere, since many emsdk users don't have any of those tools installed.

Is there some reason why the freetype is hard to maintain in its current form?

[HCLJason]

The port is version 2.6.1, which doesn't have a bunch of security fixes.

https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=freetype

I see a ninja-using PR into tools/ports here: #17809

[sbc100]

Yes we are considering adding ninja as a dependency, but have not actually taken that step.

Note that with that PR the existing ports build system will generate ninja directly (we still don't plan on shipping cmake).

Is there some issue with upgrading freetype using the current (internal) build system for ports?

[sbc100]

If you goal is to just to upgrade freetype I think we can probably just do that without adding support for external built tools.

We can also discuses the benefits of using an external build tool separately.

[HCLJason]

I tried that. The initial issue is duplicate zlib symbols, so naturally I swapped the compile to use the system zlib.

I tried both using it as a compiler flag ('-DFT_REQUIRE_ZLIB') and adding it to the ftoption.h file:
with open(os.path.join(source_path, 'include', 'freetype', 'config','ftoption.h'), 'a') as ftheader:
ftheader.write('#define FT_CONFIG_OPTION_SYSTEM_ZLIB')

At this point, I don't think this needs to get fixed, but I'll try to get an updated zlib + freetype port for you soon.

[sbc100]

Thanks, if need help with the update please reach out. Perhaps it might be worth opening a bug to update freetype? (and we can close this one)

[HCLJason]

I'll open up an issue once I have the ports ready to go.

For the freetype update, there's 3 ftsystem files - unix, win32, and base. I assume that I want base, right?

Where did you get the ftconf.h file? It looks like a mashup of ftoption.h from emconfigure configure and something else.

[sbc100]

I imagine you will want base + unix

[HCLJason]

Using unix has issues, but base works fine. I have PRs in for both zlib and freetype updates.

#18087
#18088

[Oipo]

I think I'm running into a related issue with the standard ports SDL_ttf/SDL_mixer, so I'll attach my log here.

emscripten cache issue.txt