[Question] Emscripten include paths missing in Clang WebAssembly toolchain ?

[anutosh491]

Hey @sbc100 , I have a simple question that possibly you could clarify for me

Question : Should Clang’s WebAssembly toolchain handle Emscripten-specific include paths like /include/compat when targeting wasm32-unknown-emscripten?

When compiling Emscripten-targeted code with emcc, the driver invokes Clang with additional Emscripten-specific include paths.
For example, compiling a trivial file:

emcc -v b.cpp -o b.html

// prints
/Users/.../clang -target wasm32-unknown-emscripten \
  -fignore-exceptions \
  -mllvm -combiner-global-alias-analysis=false \
  -mllvm -enable-emscripten-sjlj \
  -mllvm -disable-lsr \
  --sysroot=/Users/.../emscripten/cache/sysroot \
  -DEMSCRIPTEN \
  -Xclang -iwithsysroot/include/fakesdl \
  -Xclang -iwithsysroot/include/compat \
  -v -c b.cpp -o ...

The cc1 command as a result shows

"/Users/anutosh491/work/emsdk/upstream/bin/clang-22" -cc1 -triple wasm32-unknown-emscripten -..... -resource-dir /Users/anutosh491/work/emsdk/upstream/lib/clang/22 -D EMSCRIPTEN -isysroot /Users/anutosh491/work/emsdk/upstream/emscripten/cache/sysroot -internal-isystem /Users/anutosh491/work/emsdk/upstream/emscripten/cache/sysroot/include/wasm32-emscripten/c++/v1 -internal-isystem /Users/anutosh491/work/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1 -internal-isystem /Users/anutosh491/work/emsdk/upstream/lib/clang/22/include -internal-isystem /Users/anutosh491/work/emsdk/upstream/emscripten/cache/sysroot/include/wasm32-emscripten -internal-isystem /Users/anutosh491/work/emsdk/upstream/emscripten/cache/sysroot/include ...... -iwithsysroot/include/fakesdl -iwithsysroot/include/compat -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -o /var/folders/m1/cdn74f917994jd99d_2cpf440000gn/T/emscripten_temp_2brpcg03/b.o -x c++ b.cpp

This ensures that headers such as <xlocale.h> (under /include/compat) are found correctly.

However, when invoking Clang directly (or using clang-repl / embedded use cases) with the same target:

clang --target=wasm32-unknown-emscripten -v -c b.cpp

the default cc1 invocation does not include those -iwithsysroot/include/... directories, and headers like <xlocale.h> are not found.

Looking at clang/lib/Driver/ToolChains/WebAssembly.cpp
there is no explicit handling for /include/compat or other Emscripten SDK-specific paths.
This toolchain is generic for WebAssembly and not Emscripten-specific, although it already handles Emscripten-specific backend flags such as:

-mllvm -enable-emscripten-cxx-exceptions
-mllvm -enable-emscripten-sjlj

So I wanted to clarify
Should Emscripten-specific include directories (e.g. /include/compat, /include/fakesdl) be added automatically by Clang’s WebAssembly toolchain when the target triple is wasm32-unknown-emscripten or are they considered the responsibility of the emcc wrapper?

Essentially:

i) clang --target=wasm32-unknown-emscripten → should it behave the same as emcc, or

ii) should users explicitly pass these -iwithsysroot flags when using Clang standalone?

[anutosh491]

Aksing cause I thought the headers under compat might be handled through the clang toolchain for webassembly/emscripten itself but I saw this in my wasm based C++ Jupyter Kernel . (https://compiler-research.org/xeus-cpp/lab/index.html ... feel free to check out the cc1 command by inspecting the console)

obvioulsy if I pass the flags -iwithsysroot ... path/to/compat to the kernel that does solve things

But i was curious if the toolchain should do this itself or is it expected to be done on the user side ?

[anutosh491]

Basically I can see the toolchain is webassembly target-generic and tool emscripten specific as such but I do see some emscripten EH handling stuff so I thought things like this can also be looked into probably ?!

[sbc100]

As you can see right now emscripten itself is acting as the compiler driver add adding flags like this to the clang commands.

I do think it would be nice if we could have clang do as much as this possible but those directories like fakesdl and compat are only conditionally added by emcc under certain circumstance. i.e we do not unconditionally add them.

In short, I think if are you using bare clang like this you will continue to have to replicatate/reproduce some of the logic and command line flags from the emcc driver. At least for now.

[anutosh491]

Thanks for your reply. Shall close the above issue soon, if I think all my doubts have been answered.

[anutosh491]

but those directories like fakesdl and compat are only conditionally added by emcc under certain circumstance. i.e we do not unconditionally add them.

ohh could you educate me on what these conditions are ?
Asking cause I see them being for simple operations like emcc -v b.cpp -o b.html

I do think it would be nice if we could have clang do as much as this possible

Do you think it would make sense to extend the WebAssembly toolchain (WebAssembly.cpp) or build on top of it, to support a Emscripten specific toolchain ? maybe Emscripten.cpp ? (Would like to help out if this was ever considered !)

[sbc100]

but those directories like fakesdl and compat are only conditionally added by emcc under certain circumstance. i.e we do not unconditionally add them.

ohh could you educate me on what these conditions are ? Asking cause I see them being for simple operations like emcc -v b.cpp -o b.html

The logic for choosing which extra compile flag to pass is largely contained in this file: https://github.com/emscripten-core/emscripten/blob/main/tools/compile.py.

I've tried over years to reduce this as much as possible so that bare clang can be used in many cases.. but you'd still need to replicate all this logic if you want to reproduce the behavior of emcc exactly.

I do think it would be nice if we could have clang do as much as this possible

Do you think it would make sense to extend the WebAssembly toolchain (WebAssembly.cpp) or build on top of it, to support a Emscripten specific toolchain ? maybe Emscripten.cpp ? (Would like to help out if this was ever considered !)

Yes, but its tricky because we have a lot of -s settings and clang itself does not have parser for these. In the long run that is the goal, but we need to move thoughtfully and carefully towards it I think.

[anutosh491]

Yes, but its tricky because we have a lot of -s settings and clang itself does not have parser for these

Yes, I was just thinking about this. Would possibly introduce quite some overlap (from emscripten's source) if we try to address these ?

In the long run that is the goal, but we need to move thoughtfully and carefully towards it I think.

Yupp, if there's something minimalistic we could start off with, that you might have thought off .... I would surely give it an attempt !

[anutosh491]

I was also playing around with the wasm exception handling models and I can see some differences there too (not sure if intended or not)

Understanding -fwasm-exceptions

1. With bare clang I see it maps to -mllvm -wasm-enable-eh which is what is expected right

anutosh491@Anutoshs-MacBook-Air clang-wasm-repl % /Users/anutosh491/work/emsdk/upstream/bin/clang++ \
    -target wasm32-unknown-emscripten \
    -fwasm-exceptions \
    -v \
    -c test.cpp \
    -o test.o

clang version 22.0.0git (https:/github.com/llvm/llvm-project c13ac9cadf1f9b4fa886b82d1e84a5feb0439023)
Target: wasm32-unknown-emscripten
Thread model: posix
InstalledDir: /Users/anutosh491/work/emsdk/upstream/bin
 (in-process)

"/Users/anutosh491/work/emsdk/upstream/bin/clang-22" -cc1 -triple wasm32-unknown-emscripten -emit-obj -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name test.cpp -mrelocation-model static -mframe-pointer=none -ffp-contract=on -fno-rounding-math -mconstructor-aliases -target-feature +exception-handling -target-feature +multivalue -target-feature +reference-types -exception-model=wasm -mllvm -wasm-enable-eh -target-cpu generic -fvisibility=hidden -debugger-tuning=gdb -fdebug-compilation-dir=/Users/anutosh491/work/clang-wasm-repl -v -fcoverage-compilation-dir=/Users/anutosh491/work/clang-wasm-repl -resource-dir /Users/anutosh491/work/emsdk/upstream/lib/clang/22 -internal-isystem /Users/anutosh491/work/emsdk/upstream/lib/clang/22/include -internal-isystem /include/wasm32-emscripten -internal-isystem /include -fdeprecated-macro -ferror-limit 19 -fmessage-length=157 -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fcxx-exceptions -fexceptions -exception-model=wasm -fcolor-diagnostics -o test.o -x c++ test.cpp

2. With emcc

step 1 : activate 4.0.18
step 2 : check cflags

anutosh491@Anutoshs-MacBook-Air emsdk % emcc --cflags -target wasm32-unknown-emscripten -fignore-exceptions -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --sysroot=/Users/anutosh491/work/emsdk/upstream/emscripten/cache/sysroot -DEMSCRIPTEN -Xclang -iwithsysroot/include/fakesdl -Xclang -iwithsysroot/include/compat

step 3 : improve EMCC_CFLAGS

anutosh491@Anutoshs-MacBook-Air emsdk % export EXCEPTION_HANDLING_FLAGS="-fwasm-exceptions" 
anutosh491@Anutoshs-MacBook-Air emsdk % export EMCC_CFLAGS="${EXCEPTION_HANDLING_FLAGS}"

step 4 : check clags again

anutosh491@Anutoshs-MacBook-Air emsdk % emcc --cflags -target wasm32-unknown-emscripten -mllvm -combiner-global-alias-analysis=false -mllvm -wasm-enable-sjlj -mllvm -wasm-use-legacy-eh -mllvm -disable-lsr --sysroot=/Users/anutosh491/work/emsdk/upstream/emscripten/cache/sysroot -DEMSCRIPTEN -Xclang -iwithsysroot/include/fakesdl -Xclang -iwithsysroot/include/compat

Not sure why this invoked -mllvm -wasm-enable-sjlj -mllvm -wasm-use-legacy-eh ?

step 5 : Compile test.cpp

Here although I see -mllvm -wasm-enable-eh at the top probably superseding the legacy-eh at the bottom, not sure why its added in the first place ?

anutosh491@Anutoshs-MacBook-Air clang-wasm-repl % emcc test.cpp -std=c++20 -o test.o -v

/Users/anutosh491/work/emsdk/upstream/bin/clang -target wasm32-unknown-emscripten -mllvm -combiner-global-alias-analysis=false -mllvm -wasm-enable-sjlj -mllvm -wasm-use-legacy-eh -mllvm -disable-lsr --sysroot=/Users/anutosh491/work/emsdk/upstream/emscripten/cache/sysroot -DEMSCRIPTEN -Xclang -iwithsysroot/include/fakesdl -Xclang -iwithsysroot/include/compat -std=c++20 -v -fwasm-exceptions -c test.cpp -o /var/folders/m1/cdn74f917994jd99d_2cpf440000gn/T/emscripten_temp_g0csgxkm/test.o clang version 22.0.0git (https:/github.com/llvm/llvm-project c13ac9cadf1f9b4fa886b82d1e84a5feb0439023) Target: wasm32-unknown-emscripten Thread model: posix InstalledDir: /Users/anutosh491/work/emsdk/upstream/bin (in-process) "/Users/anutosh491/work/emsdk/upstream/bin/clang-22" -cc1 -triple wasm32-unknown-emscripten -emit-obj -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name test.cpp -mrelocation-model static -mframe-pointer=none -ffp-contract=on -fno-rounding-math -mconstructor-aliases -target-feature +exception-handling -target-feature +multivalue -target-feature +reference-types -exception-model=wasm -mllvm -wasm-enable-eh -target-cpu generic -fvisibility=hidden -debugger-tuning=gdb -fdebug-compilation-dir=/Users/anutosh491/work/clang-wasm-repl -v -fcoverage-compilation-dir=/Users/anutosh491/work/clang-wasm-repl -resource-dir /Users/anutosh491/work/emsdk/upstream/lib/clang/22 -D EMSCRIPTEN -isysroot /Users/anutosh491/work/emsdk/upstream/emscripten/cache/sysroot -internal-isystem /Users/anutosh491/work/emsdk/upstream/emscripten/cache/sysroot/include/wasm32-emscripten/c++/v1 -internal-isystem /Users/anutosh491/work/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1 -internal-isystem /Users/anutosh491/work/emsdk/upstream/lib/clang/22/include -internal-isystem /Users/anutosh491/work/emsdk/upstream/emscripten/cache/sysroot/include/wasm32-emscripten -internal-isystem /Users/anutosh491/work/emsdk/upstream/emscripten/cache/sysroot/include -std=c++20 -fdeprecated-macro -ferror-limit 19 -fmessage-length=157 -fgnuc-version=4.2.1 -fno-implicit-modules -fskip-odr-check-in-gmf -fcxx-exceptions -fexceptions -exception-model=wasm -fcolor-diagnostics -iwithsysroot/include/fakesdl -iwithsysroot/include/compat -mllvm -combiner-global-alias-analysis=false -mllvm -wasm-enable-sjlj -mllvm -wasm-use-legacy-eh -mllvm -disable-lsr -o /var/folders/m1/cdn74f917994jd99d_2cpf440000gn/T/emscripten_temp_g0csgxkm/test.o -x c++ test.cpp

[sbc100]

The logic for choosing those eh flags is here: https://github.com/emscripten-core/emscripten/blob/5b42968e8aff366a0967a9a8fd9de10e12e985db/tools/building.py#L312-L319

And here: https://github.com/emscripten-core/emscripten/blob/5b42968e8aff366a0967a9a8fd9de10e12e985db/tools/building.py#L93-L125

Note that WASM_LEGACY_EXCEPTIONS defaults to true so you can expect to see -wasm-use-legacy-eh by default, even with -fwasm-exceptions.

[anutosh491]

Closing for now. Shall reopen if I have more doubts on this front !