wasm-ld sometimes produces corrupt output in __wasm_apply_data_relocs

[jechter]

In our project (using NativeAOT-LLVM combined with emscripten) we have a recurring instability where sometimes wasm-ld will produce invalid wasm binary data, in the __wasm_apply_data_relocs function. Reproduction is tricky, because it will not consistently happen with the same inputs. But I have recently found a relatively stable repro I consider worth reporting.

I attached a zip file with the repro project. On Linux x64, running the following commands will reproduce the issue (I have been unable to reproduce it on a mac-arm64, or on linux-arm64):

unzip repro.zip
cd e947e68e8f4fe7846f3e19aee1b89004
mkdir out
wasm-ld  -o out/libHelloWorldGame-ilc.wasm --whole-archive -L~/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/pic System.Runtime.a HelloWorldGame_ExportModuleLoadFunction.a libSystem.Runtime.Shared.wasm libCuriosityCore.wasm libEntities.wasm libHotReload.wasm libMathematics.wasm libMessaging.wasm libRaylibRenderer.wasm libExternalRaylib.wasm libBlobs-ilc.wasm libCuriosityCore-ilc.wasm libEntities-ilc.wasm libHotReload-ilc.wasm libMathematics-ilc.wasm libRaylibRenderer-ilc.wasm libSystem.Runtime-ilc.wasm emscripten_temp/ilc_output.0_0.o emscripten_temp/ilc_output.10_1.o emscripten_temp/ilc_output.11_2.o emscripten_temp/ilc_output.12_3.o emscripten_temp/ilc_output.13_4.o emscripten_temp/ilc_output.14_5.o emscripten_temp/ilc_output.15_6.o emscripten_temp/ilc_output.1_7.o emscripten_temp/ilc_output.2_8.o emscripten_temp/ilc_output.3_9.o emscripten_temp/ilc_output.4_10.o emscripten_temp/ilc_output.5_11.o emscripten_temp/ilc_output.6_12.o emscripten_temp/ilc_output.7_13.o emscripten_temp/ilc_output.8_14.o emscripten_temp/ilc_output.9_15.o emscripten_temp/ilc_output.debug_16.o 69d11ca8b1fd28af0835bb0ea36410d2/ilc_output.external.o 69d11ca8b1fd28af0835bb0ea36410d2/ilc_output.o --no-whole-archive -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --import-memory --export-dynamic --export=__wasm_call_ctors --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__start_em_lib_deps --export-if-defined=__stop_em_lib_deps --export-if-defined=__start_em_js --export-if-defined=__stop_em_js --export-if-defined=main --export-if-defined=__main_argc_argv --export-if-defined=__wasm_apply_data_relocs --export-if-defined=fflush --experimental-pic --unresolved-symbols=import-dynamic -shared --stack-first

Then trying to use the produced wasm file (out/libHelloWorldGame-ilc.wasm) will usually fail. Eg, running wasm-tools print out/libHelloWorldGame-ilc.wasm will output something like this:

(module $libHelloWorldGame-ilc.wasm
  (@dylink.0
    (mem-info (memory 3932 3))
    (needed "libSystem.Runtime.Shared.wasm" "libCuriosityCore.wasm" "libEntities.wasm" "libHotReload.wasm" "libMathematics.wasm" "libMessaging.wasm" "libRaylibRenderer.wasm" "libExternalRaylib.wasm" "libBlobs-ilc.wasm" "libCuriosityCore-ilc.wasm" "libEntities-ilc.wasm" "libHotReload-ilc.wasm" "libMathematics-ilc.wasm" "libRaylibRenderer-ilc.wasm" "libSystem.Runtime-ilc.wasm")
  )
[...]
  (func $__wasm_apply_data_relocs (;68;) (type 2)
[...]
    global.get $HelloWorldGame_Generated_SystemRegistration__TestGameSystem_Execute$F0_Filter
    i32.store
    i32.const 1387
    global.get $__memory_base
    i32.add
error: invalid var_i32: integer too large (at offset 0xbeb6)

This does not repro 100% for me, but most of the time (but only on linux-x64 - we've seen this issue on macs, but much rarer). If it does not repro, try just running the wasm-ld command again.

I believe this to be a bug in wasm-ld. It is possible that the inputs are somehow malformed (given that NativeAOT-LLVM is also an unsupported compiler technology), but then I would expect wasm-ld to report an error, or at least fail consistently, but not to randomly produce invalid wasm.

repro.zip

[SingleAccretion]

This is another variant of #146923, the callstack is:

Exception Code: 0x80000003
 #0 0x00007ff6bde9dc81 std::_Optional_construct_base<unsigned int>::operator*(void) const & C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\include\optional:192:0
 #1 0x00007ff6be589c0d lld::wasm::InputFunction::getFunctionIndex(void) const C:\Users\Accretion\source\dotnet\llvm-project\lld\wasm\InputChunks.h:293:0
 #2 0x00007ff6be58456f lld::wasm::FunctionSymbol::getFunctionIndex(void) const C:\Users\Accretion\source\dotnet\llvm-project\lld\wasm\Symbols.cpp:238:0
 #3 0x00007ff6be58cb4e lld::wasm::ObjFile::calcNewValue(struct llvm::wasm::WasmRelocation const &, unsigned __int64, class lld::wasm::InputChunk const *) const C:\Users\Accretion\source\dotnet\llvm-project\lld\wasm\InputFiles.cpp:201:0
 #4 0x00007ff6be5b030c lld::wasm::InputChunk::generateRelocationCode(class llvm::raw_ostream &) const C:\Users\Accretion\source\dotnet\llvm-project\lld\wasm\InputChunks.cpp:448:0
 #5 0x00007ff6be5bc037 lld::wasm::`anonymous namespace'::Writer::createApplyDataRelocationsFunction C:\Users\Accretion\source\dotnet\llvm-project\lld\wasm\Writer.cpp:1466:0
 #6 0x00007ff6be5ba4b8 lld::wasm::`anonymous namespace'::Writer::createSyntheticInitFunctions C:\Users\Accretion\source\dotnet\llvm-project\lld\wasm\Writer.cpp:1140:0
 #7 0x00007ff6be5b9891 lld::wasm::`anonymous namespace'::Writer::run C:\Users\Accretion\source\dotnet\llvm-project\lld\wasm\Writer.cpp:1771:0
 #8 0x00007ff6be5b92ee lld::wasm::writeResult(void) C:\Users\Accretion\source\dotnet\llvm-project\lld\wasm\Writer.cpp:1888:0
 #9 0x00007ff6be562895 lld::wasm::`anonymous namespace'::LinkerDriver::linkerMain C:\Users\Accretion\source\dotnet\llvm-project\lld\wasm\Driver.cpp:1502:0
#10 0x00007ff6be563c4a lld::wasm::link(class llvm::ArrayRef<char const *>, class llvm::raw_ostream &, class llvm::raw_ostream &, bool, bool) C:\Users\Accretion\source\dotnet\llvm-project\lld\wasm\Driver.cpp:143:0
#11 0x00007ff6bdb3e4be lld::unsafeLldMain(class llvm::ArrayRef<char const *>, class llvm::raw_ostream &, class llvm::raw_ostream &, class llvm::ArrayRef<struct lld::DriverDef>, bool) C:\Users\Accretion\source\dotnet\llvm-project\lld\Common\DriverDispatcher.cpp:163:0
#12 0x00007ff6bd95dfa2 lld_main(int, char **, struct llvm::ToolContext const &) C:\Users\Accretion\source\dotnet\llvm-project\lld\tools\lld\lld.cpp:89:0
#13 0x00007ff6bd963256 main C:\Users\Accretion\source\dotnet\llvm-project\build-debug\tools\lld\tools\lld\lld-driver.cpp:17:0

[llvmbot]

@llvm/issue-subscribers-lld-wasm

Author: jonas echterhoff (jechter)

In our project (using NativeAOT-LLVM combined with emscripten) we have a recurring instability where sometimes wasm-ld will produce invalid wasm binary data, in the __wasm_apply_data_relocs function. Reproduction is tricky, because it will not consistently happen with the same inputs. But I have recently found a relatively stable repro I consider worth reporting.

I attached a zip file with the repro project. On Linux x64, running the following commands will reproduce the issue (I have been unable to reproduce it on a mac-arm64, or on linux-arm64):

unzip repro.zip
cd e947e68e8f4fe7846f3e19aee1b89004
mkdir out
wasm-ld  -o out/libHelloWorldGame-ilc.wasm --whole-archive -L~/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/pic System.Runtime.a HelloWorldGame_ExportModuleLoadFunction.a libSystem.Runtime.Shared.wasm libCuriosityCore.wasm libEntities.wasm libHotReload.wasm libMathematics.wasm libMessaging.wasm libRaylibRenderer.wasm libExternalRaylib.wasm libBlobs-ilc.wasm libCuriosityCore-ilc.wasm libEntities-ilc.wasm libHotReload-ilc.wasm libMathematics-ilc.wasm libRaylibRenderer-ilc.wasm libSystem.Runtime-ilc.wasm emscripten_temp/ilc_output.0_0.o emscripten_temp/ilc_output.10_1.o emscripten_temp/ilc_output.11_2.o emscripten_temp/ilc_output.12_3.o emscripten_temp/ilc_output.13_4.o emscripten_temp/ilc_output.14_5.o emscripten_temp/ilc_output.15_6.o emscripten_temp/ilc_output.1_7.o emscripten_temp/ilc_output.2_8.o emscripten_temp/ilc_output.3_9.o emscripten_temp/ilc_output.4_10.o emscripten_temp/ilc_output.5_11.o emscripten_temp/ilc_output.6_12.o emscripten_temp/ilc_output.7_13.o emscripten_temp/ilc_output.8_14.o emscripten_temp/ilc_output.9_15.o emscripten_temp/ilc_output.debug_16.o 69d11ca8b1fd28af0835bb0ea36410d2/ilc_output.external.o 69d11ca8b1fd28af0835bb0ea36410d2/ilc_output.o --no-whole-archive -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --import-memory --export-dynamic --export=__wasm_call_ctors --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__start_em_lib_deps --export-if-defined=__stop_em_lib_deps --export-if-defined=__start_em_js --export-if-defined=__stop_em_js --export-if-defined=main --export-if-defined=__main_argc_argv --export-if-defined=__wasm_apply_data_relocs --export-if-defined=fflush --experimental-pic --unresolved-symbols=import-dynamic -shared --stack-first

Then trying to use the produced wasm file (out/libHelloWorldGame-ilc.wasm) will usually fail. Eg, running wasm-tools print out/libHelloWorldGame-ilc.wasm will output something like this:

(module $libHelloWorldGame-ilc.wasm
  (@<!-- -->dylink.0
    (mem-info (memory 3932 3))
    (needed "libSystem.Runtime.Shared.wasm" "libCuriosityCore.wasm" "libEntities.wasm" "libHotReload.wasm" "libMathematics.wasm" "libMessaging.wasm" "libRaylibRenderer.wasm" "libExternalRaylib.wasm" "libBlobs-ilc.wasm" "libCuriosityCore-ilc.wasm" "libEntities-ilc.wasm" "libHotReload-ilc.wasm" "libMathematics-ilc.wasm" "libRaylibRenderer-ilc.wasm" "libSystem.Runtime-ilc.wasm")
  )
[...]
  (func $__wasm_apply_data_relocs (;68;) (type 2)
[...]
    global.get $HelloWorldGame_Generated_SystemRegistration__TestGameSystem_Execute$F0_Filter
    i32.store
    i32.const 1387
    global.get $__memory_base
    i32.add
error: invalid var_i32: integer too large (at offset 0xbeb6)

This does not repro 100% for me, but most of the time (but only on linux-x64 - we've seen this issue on macs, but much rarer). If it does not repro, try just running the wasm-ld command again.

I believe this to be a bug in wasm-ld. It is possible that the inputs are somehow malformed (given that NativeAOT-LLVM is also an unsupported compiler technology), but then I would expect wasm-ld to report an error, or at least fail consistently, but not to randomly produce invalid wasm.

repro.zip

[sbc100]

Is it possible you have a very large amount of static data? Like more than 2gb or 4gb?

[SingleAccretion]

Is it possible you have a very large amount of static data? Like more than 2gb or 4gb?

This is a(nother) manifestation of the R_WASM_FUNCTION_INDEX_I32 (or any other unusual reloc type) + -pic problem, it's not related to the data size. We could close this as duplicate of #146923.

[sbc100]

Ah yes, confirmed this is a dup of #146923.