Doorstop breaks output redirection in sh

[arrowmaster]

I've been trying to track down why exactly UnityDoorstop is breaking SteamLinuxRuntime on linux requires being injected later in the game launching process. I believed it to be because error messages were being output to stdout but that turned out to be false. I still don't know the cause but I have found a way to test it after digging more into SteamLinuxRuntime_soldier/_v2-entry-point and why it fails to run correctly.

My goal was to hopefully track down and solve why a special handling for SteamLinuxRuntime is needed in run.sh and simplify the launch script.

The following script shows that subshell output in sh is no longer being returned correctly and is instead going directly to stdout. In the final example the echo in the subshell is sent directly to stdout and x is unset.

#!/bin/sh
set -eu
a="/$0"; a=${a%/*}; a=${a#/}; a=${a:-.}; BASEDIR=$(cd "$a" || exit; pwd -P)

sh -s <<'EOF'
x="$(echo "one")"
printf "%s\n" "[$$]: x= $x"
EOF

export LD_PRELOAD="$BASEDIR/libdoorstop.so"

sh -s <<'EOF'
x="$(echo "two")"
printf "%s\n" "[$$]: x= $x"
EOF

export DOORSTOP_ENABLED=1

sh -s <<'EOF'
x="$(echo "three")"
printf "%s\n" "[$$]: x= $x"
EOF

❯ ./test-doorstop.sh
[970940]: x= one
[Doorstop] DOORSTOP_ENABLED: 0
[Doorstop] DOORSTOP_REDIRECT_OUTPUT_LOG: 0
[Doorstop] DOORSTOP_IGNORE_DISABLED_ENV: 0
[Doorstop] DOORSTOP_MONO_DEBUG_ENABLED: 0
[Doorstop] DOORSTOP_MONO_DEBUG_SUSPEND: 0
[Doorstop] DOORSTOP_MONO_DEBUG_ADDRESS: 127.0.0.1:10000
[Doorstop] DOORSTOP_TARGET_ASSEMBLY: (null)
[Doorstop] DOORSTOP_BOOT_CONFIG_OVERRIDE: (null)
[Doorstop] DOORSTOP_MONO_DLL_SEARCH_PATH_OVERRIDE: 
[Doorstop] DOORSTOP_CLR_RUNTIME_CORECLR_PATH: (null)
[Doorstop] DOORSTOP_CLR_CORLIB_DIR: (null)
[Doorstop] Doorstop not enabled! Skipping!
[970942]: x= two
[Doorstop] DOORSTOP_ENABLED: 1
[Doorstop] DOORSTOP_REDIRECT_OUTPUT_LOG: 0
[Doorstop] DOORSTOP_IGNORE_DISABLED_ENV: 0
[Doorstop] DOORSTOP_MONO_DEBUG_ENABLED: 0
[Doorstop] DOORSTOP_MONO_DEBUG_SUSPEND: 0
[Doorstop] DOORSTOP_MONO_DEBUG_ADDRESS: 127.0.0.1:10000
[Doorstop] DOORSTOP_TARGET_ASSEMBLY: (null)
[Doorstop] DOORSTOP_BOOT_CONFIG_OVERRIDE: (null)
[Doorstop] DOORSTOP_MONO_DLL_SEARCH_PATH_OVERRIDE: 
[Doorstop] DOORSTOP_CLR_RUNTIME_CORECLR_PATH: (null)
[Doorstop] DOORSTOP_CLR_CORLIB_DIR: (null)
[Doorstop] Failed to hook dlsym, ignoring it. Error: no such function: dlsym
[Doorstop] Failed to hook fclose, ignoring it. Error: no such function: fclose
three
[970944]: x=