[WIP] Experimental WebAssembly Support

[1div0]

emcmake cmake -B WebAssembly
cmake --build WebAssembly

Exiv2 WebAssembly.log

Failure by linking stage. Need to dig deeper.

[ghost]

👇 Click on the image for a new way to code review

Legend

[neheb]

Uhhh try with meson instead

edit: need to pass -Ddefault_library=static

ERROR: ld.wasm does not support shared libraries.

edit2: nvm:

em++: error: lib_convert.a: Unknown format, not a static library!

[1div0]

meson setup --cross-file wasm32.text WebAssembly
ninja -C WebAssembly

ninja: Entering directory `WebAssembly'
[1/1] Linking target exiv2
FAILED: exiv2
em++ -o exiv2 exiv2.p/app_actions.cpp.o exiv2.p/app_app_utils.cpp.o exiv2.p/app_exiv2.cpp.o exiv2.p/app_getopt.cpp.o -sERROR_ON_UNDEFINED_SYMBOLS=1 -Wl,--start-group libexiv2.a -Wl,--end-group
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::indent(unsigned long)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::indent(unsigned long)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::indent(unsigned long)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::indent(unsigned long)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::TiffMapping::findDecoder(std::__2::basic_string<char, std::__2::char_traits, std::__2::allocator> const&, unsigned int, Exiv2::IfdId)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::TiffParserWorker::decode(Exiv2::ExifData&, Exiv2::IptcData&, Exiv2::XmpData&, unsigned char const*, unsigned long, unsigned int, void (Exiv2::Internal::TiffDecoder::* ()(std::__2::basic_string<char, std::__2::char_traits, std::__2::allocator> const&, unsigned int, Exiv2::IfdId))(Exiv2::Internal::TiffEntryBase const), Exiv2::Internal::TiffHeaderBase*)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::TiffMapping::findDecoder(std::__2::basic_string<char, std::__2::char_traits, std::__2::allocator> const&, unsigned int, Exiv2::IfdId)
wasm-ld: error: libexiv2.a(src_bmffimage.cpp.o): undefined symbol: Exiv2::Internal::TiffParserWorker::decode(Exiv2::ExifData&, Exiv2::IptcData&, Exiv2::XmpData&, unsigned char const*, unsigned long, unsigned int, void (Exiv2::Internal::TiffDecoder::* ()(std::__2::basic_string<char, std::__2::char_traits, std::__2::allocator> const&, unsigned int, Exiv2::IfdId))(Exiv2::Internal::TiffEntryBase const), Exiv2::Internal::TiffHeaderBase*)
wasm-ld: error: libexiv2.a(src_basicio.cpp.o): undefined symbol: Exiv2::Internal::stringFormat(char const*, ...)
wasm-ld: error: libexiv2.a(src_exif.cpp.o): undefined symbol: Exiv2::Internal::printValue(std::__2::basic_ostream<char, std::__2::char_traits>&, Exiv2::Value const&, Exiv2::ExifData const*)
wasm-ld: error: libexiv2.a(src_exif.cpp.o): undefined symbol: Exiv2::Internal::tagInfo(unsigned short, Exiv2::IfdId)
wasm-ld: error: libexiv2.a(src_exif.cpp.o): undefined symbol: Exiv2::Internal::TiffHeader::TiffHeader(Exiv2::ByteOrder, unsigned int, bool)
wasm-ld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)
em++: error: '/2TB/usr/src/github.com/emscripten-core/emsdk/upstream/bin/wasm-ld -o exiv2.wasm exiv2.p/app_actions.cpp.o exiv2.p/app_app_utils.cpp.o exiv2.p/app_exiv2.cpp.o exiv2.p/app_getopt.cpp.o libexiv2.a -L/2TB/usr/src/github.com/emscripten-core/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten -lGL -lal -lhtml5 -lstubs-debug -lnoexit -lc-debug -ldlmalloc -lcompiler_rt -lc++-noexcept -lc++abi-debug-noexcept -lsockets -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --allow-undefined-file=/tmp/tmptp3dz3hq.undefined --strip-debug --export-if-defined=main --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__start_em_lib_deps --export-if-defined=__stop_em_lib_deps --export-if-defined=__start_em_js --export-if-defined=__stop_em_js --export-if-defined=__main_argc_argv --export-if-defined=fflush --export=emscripten_stack_get_end --export=emscripten_stack_get_free --export=emscripten_stack_get_base --export=emscripten_stack_get_current --export=emscripten_stack_init --export=stackSave --export=stackRestore --export=stackAlloc --export=__errno_location --export=__get_temp_ret --export=__set_temp_ret --export=__wasm_call_ctors --export=malloc --export=__cxa_is_pointer_type --export=ntohs --export=htons --export=emscripten_builtin_memalign --export-table -z stack-size=65536 --initial-memory=16777216 --no-entry --max-memory=16777216 --stack-first' failed (returned 1)
ninja: build stopped: subcommand failed.

[neheb]

That's your problem

[1div0]

em++ -o exiv2.js exiv2.p/app_actions.cpp.o exiv2.p/app_app_utils.cpp.o exiv2.p/app_exiv2.cpp.o exiv2.p/app_getopt.cpp.o libexiv2.a.p/*.o libexiv2int.a.p/*.o -sERROR_ON_UNDEFINED_SYMBOLS=1

peter.kovar@Pascal /2TB/usr/src/github.com/1div0/exiv2/WebAssembly
€ file exiv2.js exiv2.wasm
exiv2.js: ASCII text, with very long lines (357)
exiv2.wasm: WebAssembly (wasm) binary module version 0x1 (MVP)

[1div0]

[neheb]

What's the issue?

[1div0]

This was output to the developer console in Brave. Just a confirmation that Exiv2 library is executing via WebAssembly.

[neheb]

OK. The purpose of the meson build is simply to build the library. The goal is not to fully replace the CMake build. I have no idea whether or not it works properly.

[1div0]

Well, my intent was to make sure that WebAssembly can be built and viable option. Maybe even for 1.0. Thoughts?

[neheb]

Sure, why not.

I can’t help much with CMake though

[neheb]

I tried this again. Needs:

--- a/src/meson.build
+++ b/src/meson.build
@@ -115,12 +115,6 @@ if host_machine.system() == 'windows' and get_option('default_library') != 'stat
   dllapi = '-DEXIV2API=__declspec(dllimport)'
 endif
 
-cmake = import('cmake')
-cmake.write_basic_package_version_file(
-  name: meson.project_name(),
-  version: meson.project_version(),
-)
-
 pkg = import('pkgconfig')
 pkg.generate(
   exiv2,

Maybe I should completely get rid of it. It doesn't even work for cmake.

meson ran with

meson --cross-file=em.txt -Ddefault_library=static em

em.txt:

[binaries]
c = 'emcc'
cpp = 'em++'
ar = 'emar'
nm = 'emnm'

[host_machine]
system = 'emscripten'
cpu_family = 'x86_64'
cpu = 'x86_64'
endian = 'little'

[neheb]

if you want to build the subprojects as well, -Dauto_features=enable and -DXXX:default_library=static for each subproject.

[neheb]

>>> defined as (i32, i32, i32) -> i32 in subprojects/zlib-1.2.13/libz.a(gzlib.c.o)
>>> defined as (i32, i64, i32) -> i64 in /usr/share/emscripten/cache/sysroot/lib/wasm32-emscripten/libc-debug.a(lseek.o)

great, a bug in the wrap.

[neheb]

hmm tests don't compile

[212/212] Linking target unitTests/unit_tests.js
FAILED: unitTests/unit_tests.js 
em++  -o unitTests/unit_tests.js unitTests/unit_tests.js.p/test_DateValue.cpp.o unitTests/unit_tests.js.p/test_Error.cpp.o unitTests/unit_tests.js.p/test_FileIo.cpp.o unitTests/unit_tests.js.p/test_ImageFactory.cpp.o unitTests/unit_tests.js.p/test_IptcKey.cpp.o unitTests/unit_tests.js.p/test_LangAltValueRead.cpp.o unitTests/unit_tests.js.p/test_Photoshop.cpp.o unitTests/unit_tests.js.p/test_TimeValue.cpp.o unitTests/unit_tests.js.p/test_XmpKey.cpp.o unitTests/unit_tests.js.p/test_basicio.cpp.o unitTests/unit_tests.js.p/test_bmpimage.cpp.o unitTests/unit_tests.js.p/test_cr2header_int.cpp.o unitTests/unit_tests.js.p/test_datasets.cpp.o unitTests/unit_tests.js.p/test_enforce.cpp.o unitTests/unit_tests.js.p/test_futils.cpp.o unitTests/unit_tests.js.p/test_helper_functions.cpp.o unitTests/unit_tests.js.p/test_image_int.cpp.o unitTests/unit_tests.js.p/test_jp2image.cpp.o unitTests/unit_tests.js.p/test_jp2image_int.cpp.o unitTests/unit_tests.js.p/test_safe_op.cpp.o unitTests/unit_tests.js.p/test_slice.cpp.o unitTests/unit_tests.js.p/test_tiffheader.cpp.o unitTests/unit_tests.js.p/test_types.cpp.o unitTests/unit_tests.js.p/test_utils.cpp.o unitTests/unit_tests.js.p/test_asfvideo.cpp.o unitTests/unit_tests.js.p/test_matroskavideo.cpp.o unitTests/unit_tests.js.p/test_riffVideo.cpp.o unitTests/unit_tests.js.p/test_pngimage.cpp.o unitTests/unit_tests.js.p/.._subprojects_googletest-1.13.0_googletest_src_gtest-all.cc.o unitTests/unit_tests.js.p/.._subprojects_googletest-1.13.0_googlemock_src_gmock-all.cc.o unitTests/unit_tests.js.p/.._subprojects_googletest-1.13.0_googlemock_src_gmock_main.cc.o -s ERROR_ON_UNDEFINED_SYMBOLS=1 -Wl,--start-group subprojects/expat-2.5.0/libexpat.a subprojects/zlib-1.2.13/libz.a src/libexiv2.a src/lib_convert.a src/libexiv2int.a -Wl,--end-group -s USE_PTHREADS=1 -s PTHREAD_POOL_SIZE=4
wasm-ld-13: error: --shared-memory is disallowed by datasets.cpp.o because it was not compiled with 'atomics' or 'bulk-memory' features.
em++: error: '/usr/bin/wasm-ld-13 -o unitTests/unit_tests.wasm unitTests/unit_tests.js.p/test_DateValue.cpp.o unitTests/unit_tests.js.p/test_Error.cpp.o unitTests/unit_tests.js.p/test_FileIo.cpp.o unitTests/unit_tests.js.p/test_ImageFactory.cpp.o unitTests/unit_tests.js.p/test_IptcKey.cpp.o unitTests/unit_tests.js.p/test_LangAltValueRead.cpp.o unitTests/unit_tests.js.p/test_Photoshop.cpp.o unitTests/unit_tests.js.p/test_TimeValue.cpp.o unitTests/unit_tests.js.p/test_XmpKey.cpp.o unitTests/unit_tests.js.p/test_basicio.cpp.o unitTests/unit_tests.js.p/test_bmpimage.cpp.o unitTests/unit_tests.js.p/test_cr2header_int.cpp.o unitTests/unit_tests.js.p/test_datasets.cpp.o unitTests/unit_tests.js.p/test_enforce.cpp.o unitTests/unit_tests.js.p/test_futils.cpp.o unitTests/unit_tests.js.p/test_helper_functions.cpp.o unitTests/unit_tests.js.p/test_image_int.cpp.o unitTests/unit_tests.js.p/test_jp2image.cpp.o unitTests/unit_tests.js.p/test_jp2image_int.cpp.o unitTests/unit_tests.js.p/test_safe_op.cpp.o unitTests/unit_tests.js.p/test_slice.cpp.o unitTests/unit_tests.js.p/test_tiffheader.cpp.o unitTests/unit_tests.js.p/test_types.cpp.o unitTests/unit_tests.js.p/test_utils.cpp.o unitTests/unit_tests.js.p/test_asfvideo.cpp.o unitTests/unit_tests.js.p/test_matroskavideo.cpp.o unitTests/unit_tests.js.p/test_riffVideo.cpp.o unitTests/unit_tests.js.p/test_pngimage.cpp.o unitTests/unit_tests.js.p/.._subprojects_googletest-1.13.0_googletest_src_gtest-all.cc.o unitTests/unit_tests.js.p/.._subprojects_googletest-1.13.0_googlemock_src_gmock-all.cc.o unitTests/unit_tests.js.p/.._subprojects_googletest-1.13.0_googlemock_src_gmock_main.cc.o subprojects/expat-2.5.0/libexpat.a subprojects/zlib-1.2.13/libz.a src/libexiv2.a src/lib_convert.a src/libexiv2int.a -L/usr/share/emscripten/cache/sysroot/lib/wasm32-emscripten /usr/share/emscripten/cache/sysroot/lib/wasm32-emscripten/crtbegin.o -lGL-mt -lal -lhtml5 -lstubs-debug -lnoexit -lc-mt-debug -ldlmalloc-mt -lcompiler_rt-mt -lc++-mt-noexcept -lc++abi-mt-noexcept -lsockets-mt -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --import-undefined --import-memory --shared-memory --strip-debug --export-if-defined=main --export-if-defined=_emscripten_thread_init --export-if-defined=_emscripten_thread_exit --export-if-defined=_emscripten_thread_crashed --export-if-defined=emscripten_tls_init --export-if-defined=emscripten_current_thread_process_queued_calls --export-if-defined=pthread_self --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__stdio_exit --export=emscripten_stack_get_end --export=emscripten_stack_get_free --export=emscripten_stack_get_base --export=emscripten_stack_init --export=stackSave --export=stackRestore --export=stackAlloc --export=__wasm_call_ctors --export=__errno_location --export=emscripten_dispatch_to_thread_ --export=_emscripten_thread_free_data --export=_emscripten_allow_main_runtime_queued_calls --export=emscripten_main_browser_thread_id --export=emscripten_main_thread_process_queued_calls --export=emscripten_run_in_main_runtime_thread_js --export=emscripten_stack_set_limits --export=emscripten_sync_run_in_main_thread_2 --export=emscripten_sync_run_in_main_thread_4 --export=emscripten_builtin_memalign --export=malloc --export=free --export=ntohs --export=htons --export=htonl --export-table -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=16777216 --global-base=1024' failed (returned 1)
ninja: build stopped: subcommand failed

[neheb]

The reason for the tests is because gtest uses std::atomic. Might be fixed in a newer version of emscripten. Who knows.

[1div0]

Will check ASAP.

[1div0]

meson setup --cross-file wasm32.text -Ddefault_library=static WebAssembly
ninja -C WebAssembly

[codecov]

Codecov Report

Merging #2561 (8bb5636) into main (fe44c8c) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #2561   +/-   ##
=======================================
  Coverage   63.96%   63.96%           
=======================================
  Files         103      103           
  Lines       22344    22344           
  Branches    10835    10835           
=======================================
  Hits        14293    14293           
  Misses       5828     5828           
  Partials     2223     2223