Crash on macOS generating MSL with sanitizers

[abbaswasim]

Every time I run my code with address and undefined behaviour sanitisers. I get a crash in random places usually in Spirv-Cross and it doesn't go further. An example of this is below. It has taken me a a while to work this out as this is working fine in normal conditions. The reason is there is a stack-overflow happening in SPIRV-Cross's recursive MSL codegen on worker threads only. ChatGPT says.

Under ASan each stack frame is larger, and std::thread workers typically get a much smaller default stack than the main thread, so deep recursion in spirv_cross::CompilerGLSL::emit_block_chain* / CompilerMSL::to_function_args exhausts it.

I tested this theory, below is a reproducer and steps to build and run in parallel vs non-parallel modes. This happens in one of my shaders test.frag with a ridiculously long nested if/else if statements.

I guess it's probably too much to ask to remove recursions, however this means in some cases sanitisers can't be used because of this. The demo will run on linux but I can't reproduce the issue. It's MacOs specific.

Reproducer is https://github.com/abbaswasim/spv_test git clone with submodules and then:

Run in serial, works fine:
./build.sh -p false

Run in parallel, results in SO:
./build.sh -p true

There is also an spv_test_pthreads.cpp that you can see to understand how using pthreads and increasing the stack size helps.

AddressSanitizer:DEADLYSIGNAL
AddressSanitizer=================================================================
:DEADLYSIGNAL
==25550==ERROR: AddressSanitizer: stack-overflow on address 0x00016fa73de0 (pc 0x00010538b2b4 bp 0x00016fa74660 sp 0x00016fa73da0 T1)
    #0 0x00010538b2b4 in printf_common(void*, char const*, char*)+0xfc (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x1b2b4)
    #1 0x00010538bebc in vsprintf+0x64 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x1bebc)
    #2 0x00010538c6f4 in sprintf+0x38 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x1c6f4)
    #3 0x0001020fe980 in spirv_cross::convert_to_string(float, char) spirv_common.hpp:284
    #4 0x000101ec1dd4 in spirv_cross::CompilerGLSL::format_float(float) const spirv_glsl.cpp:20235
    #5 0x000101ec4dc0 in spirv_cross::CompilerGLSL::convert_float_to_string(spirv_cross::SPIRConstant const&, unsigned int, unsigned int) spirv_glsl.cpp:6383
    #6 0x000101ecbffc in spirv_cross::CompilerGLSL::constant_expression_vector(spirv_cross::SPIRConstant const&, unsigned int) spirv_glsl.cpp:6607
    #7 0x000101e3e8c8 in spirv_cross::CompilerGLSL::constant_expression(spirv_cross::SPIRConstant const&, bool, bool) spirv_glsl.cpp:6209
    #8 0x000101e67c90 in spirv_cross::CompilerGLSL::to_expression(unsigned int, bool) spirv_glsl.cpp:5641
    #9 0x000101e8ac3c in spirv_cross::CompilerGLSL::to_unpacked_expression(unsigned int, bool) spirv_glsl.cpp:5348
    #10 0x000101e92324 in spirv_cross::CompilerGLSL::to_enclosed_unpacked_expression(unsigned int, bool) spirv_glsl.cpp:5353
    #11 0x000101ee13d4 in spirv_cross::CompilerGLSL::emit_binary_op(unsigned int, unsigned int, unsigned int, unsigned int, char const*) spirv_glsl.cpp:7093
    #12 0x000101fc413c in spirv_cross::CompilerGLSL::emit_instruction(spirv_cross::Instruction const&) spirv_glsl.cpp:13640
    #13 0x00010061eac8 in spirv_cross::CompilerMSL::emit_instruction(spirv_cross::Instruction const&) spirv_msl.cpp:10700
    #14 0x000101f7a6f0 in spirv_cross::CompilerGLSL::emit_block_instructions(spirv_cross::SPIRBlock&) spirv_glsl.cpp:12190
    #15 0x0001020cf568 in spirv_cross::CompilerGLSL::emit_block_chain_inner(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18426
    #16 0x0001020b3548 in spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18267
    #17 0x0001020b9bb4 in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17811
    #18 0x0001020bb99c in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, unsigned int, spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17840
    #19 0x0001020d161c in spirv_cross::CompilerGLSL::emit_block_chain_inner(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18497
    #20 0x0001020b3548 in spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18267
    #21 0x0001020b9bb4 in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17811
    #22 0x0001020bbb8c in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, unsigned int, spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17847
    #23 0x0001020d161c in spirv_cross::CompilerGLSL::emit_block_chain_inner(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18497
    #24 0x0001020b3548 in spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18267
    #25 0x0001020b9bb4 in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17811
    #26 0x0001020bbb8c in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, unsigned int, spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17847
    #27 0x0001020d161c in spirv_cross::CompilerGLSL::emit_block_chain_inner(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18497
    #28 0x0001020b3548 in spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18267
    #29 0x0001020b9bb4 in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17811
    #30 0x0001020bbb8c in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, unsigned int, spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17847
    #31 0x0001020d161c in spirv_cross::CompilerGLSL::emit_block_chain_inner(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18497
    #32 0x0001020b3548 in spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18267
    #33 0x0001020b9bb4 in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17811
    #34 0x0001020bbb8c in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, unsigned int, spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17847
    #35 0x0001020d161c in spirv_cross::CompilerGLSL::emit_block_chain_inner(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18497
    #36 0x0001020b3548 in spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18267
    #37 0x0001020b9bb4 in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17811
    #38 0x0001020bbb8c in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, unsigned int, spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17847
    #39 0x0001020d161c in spirv_cross::CompilerGLSL::emit_block_chain_inner(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18497
    #40 0x0001020b3548 in spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18267
    #41 0x0001020b9bb4 in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17811
    #42 0x0001020bbb8c in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, unsigned int, spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17847
    #43 0x0001020d161c in spirv_cross::CompilerGLSL::emit_block_chain_inner(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18497
    #44 0x0001020b3548 in spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18267
    #45 0x0001020b9bb4 in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17811
    #46 0x0001020bbb8c in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, unsigned int, spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17847
    #47 0x0001020d161c in spirv_cross::CompilerGLSL::emit_block_chain_inner(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18497
    #48 0x0001020b3548 in spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18267
    #49 0x0001020b9bb4 in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17811
    #50 0x0001020bbb8c in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, unsigned int, spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17847
    #51 0x0001020d161c in spirv_cross::CompilerGLSL::emit_block_chain_inner(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18497
    #52 0x0001020b3548 in spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18267
    #53 0x0001020b9bb4 in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17811
    #54 0x0001020bbb8c in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, unsigned int, spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17847
    #55 0x0001020d161c in spirv_cross::CompilerGLSL::emit_block_chain_inner(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18497
    #56 0x0001020b3548 in spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18267
    #57 0x0001020b9bb4 in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17811
    #58 0x0001020bbb8c in spirv_cross::CompilerGLSL::branch(spirv_cross::TypedID<(spirv_cross::Types)6>, unsigned int, spirv_cross::TypedID<(spirv_cross::Types)6>, spirv_cross::TypedID<(spirv_cross::Types)6>) spirv_glsl.cpp:17847
    #59 0x0001020d161c in spirv_cross::CompilerGLSL::emit_block_chain_inner(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18497
    #60 0x0001020b36b4 in spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) spirv_glsl.cpp:18272
    #61 0x000101ddbbbc in spirv_cross::CompilerGLSL::emit_function(spirv_cross::SPIRFunction&, spirv_cross::Bitset const&) spirv_glsl.cpp:17585
    #62 0x000101dd838c in spirv_cross::CompilerGLSL::emit_function(spirv_cross::SPIRFunction&, spirv_cross::Bitset const&) spirv_glsl.cpp:17465
    #63 0x0001004328b4 in spirv_cross::CompilerMSL::compile() spirv_msl.cpp:1851
    #64 0x0001003aa2cc in (anonymous namespace)::compile_spirv_to_msl(std::__1::vector<unsigned int, std::__1::allocator<unsigned int>> const&) spv_test.cpp:153
    #65 0x0001003a9514 in main::$_0::operator()(unsigned long) const spv_test.cpp:251
    #66 0x0001003a927c in void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'()::operator()() const spv_test.cpp:177
    #67 0x0001003a90bc in decltype(std::declval<main::$_0 const&>()()) std::__1::__invoke[abi:ne200100]<void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'()>(main::$_0 const&) invoke.h:179
    #68 0x0001003a8fb4 in void std::__1::__thread_execute[abi:ne200100]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'()>(std::__1::tuple<main::$_0 const&, void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'()>&, std::__1::__tuple_indices<>) thread.h:205
    #69 0x0001003a81e4 in void* std::__1::__thread_proxy[abi:ne200100]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'()>>(void*) thread.h:214
    #70 0x0001053aa418 in asan_thread_start(void*)+0x4c (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x3a418)
    #71 0x00018e1ffc04 in _pthread_start+0x84 (libsystem_pthread.dylib:arm64e+0x6c04)
    #72 0x00018e1faba4 in thread_start+0x4 (libsystem_pthread.dylib:arm64e+0x1ba4)

SUMMARY: AddressSanitizer: stack-overflow spirv_common.hpp:284 in spirv_cross::convert_to_string(float, char)
Thread T1 created by T0 here:
    #0 0x0001053a59f8 in pthread_create+0x5c (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x359f8)
    #1 0x0001003a7fac in std::__1::__libcpp_thread_create[abi:ne200100](_opaque_pthread_t**, void* (*)(void*), void*) pthread.h:182
    #2 0x0001003a7c24 in std::__1::thread::thread<void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'(), 0>(main::$_0 const&) thread.h:224
    #3 0x0001003a7984 in std::__1::thread::thread<void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'(), 0>(main::$_0 const&) thread.h:219
    #4 0x0001003a7920 in main::$_0 const&* std::__1::construct_at[abi:ne200100]<std::__1::thread, void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'(), std::__1::thread*>(main::$_0 const&*, void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'()&&) construct_at.h:40
    #5 0x0001003a788c in main::$_0 const&* std::__1::__construct_at[abi:ne200100]<std::__1::thread, void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'(), std::__1::thread*>(main::$_0 const&*, void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'()&&) construct_at.h:48
    #6 0x0001003a71fc in void std::__1::allocator_traits<std::__1::allocator<std::__1::thread>>::construct[abi:ne200100]<std::__1::thread, void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'(), void, 0>(std::__1::allocator<std::__1::thread>&, main::$_0 const&*, void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'()&&) allocator_traits.h:318
    #7 0x0001003a6a78 in void std::__1::vector<std::__1::thread, std::__1::allocator<std::__1::thread>>::__construct_one_at_end[abi:ne200100]<void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'()>(void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'()&&) vector.h:742
    #8 0x0001003a192c in std::__1::thread& std::__1::vector<std::__1::thread, std::__1::allocator<std::__1::thread>>::emplace_back<void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'()>(void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const::'lambda'()&&) vector.h:1133
    #9 0x00010039515c in void (anonymous namespace)::JobSystem::run<main::$_0 const&>(main::$_0 const&) const spv_test.cpp:177
    #10 0x000100392400 in main spv_test.cpp:268
    #11 0x00018de35d50  (<unknown module>)

==25550==ABORTING

[HansKristian-Work]

Removing the recursion for nested if/else is not practical. There has been work to remove recursion for linear if/else back to back, but I don't want to ponder the complexity this would mean. If asan cannot deal with this trivial amount of recursion, it sounds more like an asan problem really.

[abbaswasim]

Probably. Not sure whether its asan or std::thread though or even MacOS threads. If there was a way to increase the stack size with std::threads that could work.

[cdavis5e]

As I recall, by default, the main thread gets 8 MiB of stack, subject to any stack-size setting in the LC_MAIN load command in the executable; but other threads only get 512 kiB. Yes, that is absurdly low; I don't know why Apple hasn't increased it for 64-bit architectures.

There is currently no way to set the initial stack size for a std::thread. If you're willing to dip down to the pthread level, you could increase it by creating a thread attributes object, and calling pthread_attr_setstacksize(). This would at least alleviate the immediate problem. Unfortunately, to my knowledge, there is no way to then create a std::thread object from the resulting pthread_t, which means that you would have to work with raw pthreads throughout.

[HansKristian-Work]

512 KiB is not aburdly low. That should be able to support 200-300+ deep recursion or something if I recall results from when I was debugging recursion for linear control flow. This is failing at like less-than 20, which means asan is likely requiring 10x the stack or something? That seems like an asan problem.

[abbaswasim]

@cdavis5e thanks for the suggestion. I provided a pthread solution in the repro as well.

There is also an spv_test_pthreads.cpp that you can see to understand how using pthreads and increasing the stack size helps.

But I think an easier workaround is to not test that shader with asan.
I would also like to add this is specific to Apple Silicon on Intel Mac I can't reproduce it.

Edit: Scratch that, reproduces on Intel Macs as well.

[cdavis5e]

512 KiB is not aburdly low.

Well, I guess it's relatively low, compared to, e.g. Windows, which gives all threads a cool 1 MiB; and glibc on Linux is even more generous, giving at least 2 MiB.