Skip to content

Commit 933c4e2

Browse files
yowlSingleAccretionjkotas
authored
[NativeAOT-LLVM] Wasm managed threads - build new nupkg (#3060)
* Add packages and CI for multithreaded debug config. * use nameSuffix in preference to new platforms. * Dont emulate for Multithreaded * pass WasmEnableThreads to build the packages to get the right name * disable PackagingTests for multithread * remove cmakeargs arg from yml and docs. * add wasmEnableThreadsArg * Remove PalGetMaximumStackBounds_MultiThreadedWasm and add pthread_attr_getstack * remove D_WASI_EMULATED_PTHREAD * Add todo for zlib-ng * put back the nameSuffix check for wasm on linux arm * dont run test for browser mt * remove wasi from multithread ci * Change CLR_CMAKE_TARGET_OS_SUBGROUP to CMAKE_USE_PTHREADS * remove atomics and bulk memory * remove wasi workaround * remove wasi pthread stubs * remove FeatureWasmPerfTracing * simplify yml use mono thread files revert naot thread files * feedback for docs disable memory growth for MT eventpipe workaround remove __cxa_thread_atexit * suppress no-pthreads-mem-growt * Update docs/workflow/building/coreclr/nativeaot.md --------- Co-authored-by: SingleAccretion <[email protected]> Co-authored-by: Jan Kotas <[email protected]>
1 parent 79c861e commit 933c4e2

File tree

15 files changed

+104
-8
lines changed

15 files changed

+104
-8
lines changed

‎docs/workflow/building/coreclr/nativeaot.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,19 @@ You should now be able to publish the project for Wasm: `dotnet publish -r brows
7676

7777
Once you build the repo, you can use the produced binaries in one of four ways specified below ("Using built binaries", "Building packages", "Convenience Visual Studio "repro" project", "Running tests").
7878

79+
## Building for Multithreaded packages
80+
81+
This is a work in progress and far from functional. Currently there exists just enough infrastructure to build packages for multithreaded runtime and libs, but they are not functional in the sense that they support multithreaded programs yet.
82+
To build the browser multithreaded packages:
83+
```
84+
build clr.aot+libs+nativeaot.packages -c Debug -a wasm -os browser '/p:WasmEnableThreads=true'
85+
```
86+
To build the runtime tests for browser
87+
```
88+
src\tests\build nativeaot Debug wasm tree nativeaot browser /p:LibrariesConfiguration=debug /p:WasmEnableThreads=true
89+
```
90+
91+
7992
### Using built binaries
8093

8194
In this workflow, you have a project file that you want to `dotnet publish`, but you want to use your own build of the compiler/runtime/framework. You need to be using a daily build of the .NET SDK downloaded from the dotnet/sdk repo. It's typically enough to just download the daily build ZIP file, unpack it, and make sure the unpacked directory is the first thing in your PATH. Don't forget to add a NuGet.config as specified by the dotnet/sdk repo- you'll hit restore issues otherwise.

‎eng/native/configurecompiler.cmake‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -148,6 +148,10 @@ elseif (CLR_CMAKE_HOST_UNIX)
148148
add_compile_options(-Wno-alloca)
149149
add_compile_options(-Wno-implicit-int-float-conversion)
150150
endif()
151+
152+
if (CMAKE_USE_PTHREADS AND CLR_CMAKE_HOST_BROWSER)
153+
add_compile_options(-pthread)
154+
endif(CMAKE_USE_PTHREADS AND CLR_CMAKE_HOST_BROWSER)
151155
endif(MSVC)
152156

153157
if (CLR_CMAKE_ENABLE_SANITIZERS)

‎eng/pipelines/runtimelab.yml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,26 @@ extends:
7171
parameters:
7272
librariesConfiguration: Debug
7373

74+
#
75+
# Build and test Wasm Debug multithreaded libraries and Debug runtime
76+
#
77+
- template: /eng/pipelines/common/platform-matrix.yml
78+
parameters:
79+
jobTemplate: /eng/pipelines/common/global-build-job.yml
80+
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
81+
buildConfig: debug
82+
platforms:
83+
- browser_wasm_win
84+
jobParameters:
85+
timeoutInMinutes: 300
86+
buildArgs: -s clr.aot+libs -c debug -rc $(_BuildConfig) '/p:WasmEnableThreads=true'
87+
nameSuffix: Multithreaded
88+
postBuildSteps:
89+
- template: /eng/pipelines/runtimelab/runtimelab-post-build-steps.yml
90+
parameters:
91+
librariesConfiguration: Debug
92+
wasmEnableThreadsArg: /p:WasmEnableThreads=true
93+
7494
#
7595
# Build and test with Debug libraries and Checked runtime
7696
#

‎eng/pipelines/runtimelab/runtimelab-post-build-steps.yml‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ parameters:
88
runtimeVariant: ''
99
isOfficialBuild: false
1010
librariesConfiguration: Debug
11+
wasmEnableThreadsArg: ''
1112

1213
steps:
1314
# For NativeAOT-LLVM, we have just built the Wasm-targeting native artifacts (the runtime and libraries).
@@ -17,7 +18,7 @@ steps:
1718
displayName: Build the ILC and RyuJit cross-compilers
1819

1920
# Build target packages (note: target libs already built).
20-
- script: $(Build.SourcesDirectory)/build$(scriptExt) nativeaot.packages -os ${{ parameters.osGroup }} -a wasm -c $(buildConfigUpper) $(_officialBuildParameter) -ci
21+
- script: $(Build.SourcesDirectory)/build$(scriptExt) nativeaot.packages -os ${{ parameters.osGroup }} -a wasm -c $(buildConfigUpper) $(_officialBuildParameter) -ci $(wasmEnableThreadsArg)
2122
displayName: Build target packages
2223

2324
# Build host packages.
@@ -26,12 +27,12 @@ steps:
2627

2728
# Build coreclr native test output outside of official build
2829
- ${{ if ne(parameters.isOfficialBuild, true) }}:
29-
- ${{ if and(eq(parameters.archType, 'wasm'), ne(parameters.nameSuffix, '')) }}:
30+
- ${{ if and(eq(parameters.archType, 'wasm'), ne(parameters.nameSuffix, ''), eq(parameters.wasmEnableThreadsArg, '')) }}:
3031
- script: pwsh $(Build.SourcesDirectory)/eng/pipelines/runtimelab/set-ilc-emulation-environment.ps1 -Arch $(hostedTargetArch)
3132
displayName: Set up ILC emulation environment
3233

3334
- ${{ if eq(parameters.archType, 'wasm') }}:
34-
- script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) nativeaot $(buildConfigUpper) ${{ parameters.osGroup }} $(crossArg) $(_officialBuildParameter) ci tree nativeaot /p:LibrariesConfiguration=${{ parameters.librariesConfiguration }}
35+
- script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) nativeaot $(buildConfigUpper) ${{ parameters.osGroup }} $(crossArg) $(_officialBuildParameter) ci tree nativeaot /p:LibrariesConfiguration=${{ parameters.librariesConfiguration }} ${{ parameters.wasmEnableThreadsArg }}
3536
displayName: Build runtime tests
3637
- ${{ else }}:
3738
- ${{ if eq(parameters.osGroup, 'windows') }}:
@@ -42,14 +43,15 @@ steps:
4243
displayName: Build runtime tests
4344

4445
- ${{ if contains(parameters.platform, 'win') }}:
45-
- script: $(Build.SourcesDirectory)/src/tests/run$(scriptExt) runnativeaottests $(buildConfigUpper) ${{ parameters.archType }} ${{ parameters.osGroup }}
46-
displayName: Run runtime tests
46+
- ${{ if eq(parameters.wasmEnableThreadsArg, '') }}:
47+
- script: $(Build.SourcesDirectory)/src/tests/run$(scriptExt) runnativeaottests $(buildConfigUpper) ${{ parameters.archType }} ${{ parameters.osGroup }}
48+
displayName: Run runtime tests
4749
- ${{ else }}:
4850
- script: $(Build.SourcesDirectory)/src/tests/run$(scriptExt) --runnativeaottests $(buildConfigUpper) ${{ parameters.archType }} ${{ parameters.osGroup }}
4951
displayName: Run runtime tests
5052

5153
# Don't compile/run the libraries tests with emulated ILC to save CI time/resources.
52-
- ${{ if and(eq(parameters.archType, 'wasm'), eq(parameters.nameSuffix, '')) }}:
54+
- ${{ if and(eq(parameters.archType, 'wasm'), eq(parameters.nameSuffix, ''), eq(parameters.wasmEnableThreadsArg, '')) }}:
5355
- script: $(Build.SourcesDirectory)/build$(scriptExt) libs.tests -test -a ${{ parameters.archType }} -os ${{ parameters.osGroup }} -lc ${{ parameters.librariesConfiguration }} -rc $(buildConfigUpper) /p:TestNativeAot=true /p:RunSmokeTestsOnly=true
5456
displayName: Build and run WebAssembly libraries tests
5557

‎src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.targets‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -438,6 +438,7 @@ The .NET Foundation licenses this file to you under the MIT license.
438438
<CompileWasmArgs Condition="'$(NativeDebugSymbols)' == 'true'">$(CompileWasmArgs) -g3</CompileWasmArgs>
439439
<CompileWasmArgs Condition="'$(WasmEnableNonTrappingFloatToIntConversions)' == 'true'">$(CompileWasmArgs) -mnontrapping-fptoint</CompileWasmArgs>
440440
<CompileWasmArgs Condition="'$(IlcLlvmExceptionHandlingModel)' == 'wasm'">$(CompileWasmArgs) -fwasm-exceptions</CompileWasmArgs>
441+
<CompileWasmArgs Condition="'$(WasmEnableThreads)' == 'true'">$(CompileWasmArgs) -pthread</CompileWasmArgs>
441442
</PropertyGroup>
442443

443444
<PropertyGroup Condition="'$(_targetOS)' == 'browser'">
@@ -604,6 +605,8 @@ The .NET Foundation licenses this file to you under the MIT license.
604605
<CustomLinkerArg Include="-s MAXIMUM_MEMORY=$(EmccMaximumHeapSize)" Condition="'$(EmccMaximumHeapSize)' != ''" />
605606
<CustomLinkerArg Include="-s INITIAL_MEMORY=$(EmccInitialHeapSize)" Condition="'$(EmccInitialHeapSize)' != ''" />
606607
<CustomLinkerArg Include="-s ASSERTIONS=1" Condition="'$(EmccEnableAssertions)' == 'true'" />
608+
<CustomLinkerArg Include="-pthread" Condition="'$(WasmEnableThreads)' == 'true'"/>
609+
<CustomLinkerArg Include="-Wno-pthreads-mem-growth" Condition="'$(WasmEnableThreads)' == 'true'"/>
607610

608611
<!-- See https://github.com/dotnet/runtimelab/issues/2357 for why we don't run wasm-opt (it would mangle our strack trace info). -->
609612
<CustomLinkerArg Condition="'$(Optimize)' == 'true'" Include="-O1" />

‎src/coreclr/nativeaot/CMakeLists.txt‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,12 @@ if(CLR_CMAKE_TARGET_OS STREQUAL wasi)
3232
add_definitions(-DTARGET_UNIX)
3333
endif(CLR_CMAKE_TARGET_OS STREQUAL wasi)
3434

35+
if((CLR_CMAKE_TARGET_OS STREQUAL wasi OR CLR_CMAKE_TARGET_OS STREQUAL emscripten)
36+
AND CMAKE_USE_PTHREADS)
37+
add_definitions(-DFEATURE_WASM_MANAGED_THREADS)
38+
endif((CLR_CMAKE_TARGET_OS STREQUAL wasi OR CLR_CMAKE_TARGET_OS STREQUAL emscripten)
39+
AND CMAKE_USE_PTHREADS)
40+
3541
if(CLR_CMAKE_TARGET_ANDROID)
3642
add_definitions(-DFEATURE_EMULATED_TLS)
3743
endif()

‎src/coreclr/nativeaot/Runtime/eventpipe/CMakeLists.txt‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,11 @@ project(Runtime)
22

33
include(FindPython)
44

5+
# Workaround from https://github.com/dotnet/runtime/pull/117513
6+
if (NOT "$ENV{EMSDK_PYTHON}" STREQUAL "")
7+
set(Python_EXECUTABLE "$ENV{EMSDK_PYTHON}")
8+
endif()
9+
510
set(CMAKE_INCLUDE_CURRENT_DIR ON)
611

712
set(EVENT_MANIFEST ${VM_DIR}/ClrEtwAll.man)

‎src/coreclr/nativeaot/System.Private.CoreLib/src/System.Private.CoreLib.csproj‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,9 @@
3535
<PropertyGroup Condition="'$(TargetsSingleThreadedWasm)' != 'true'">
3636
<FeaturePortableThreadPool>true</FeaturePortableThreadPool>
3737
<FeaturePortableTimer>true</FeaturePortableTimer>
38+
<!-- LLVM-TODO: Remove FeatureWasmManagedThreads in favor of WasmEnableThreads. -->
39+
<FeatureWasmManagedThreads Condition="'$(WasmEnableThreads)' == 'true'">true</FeatureWasmManagedThreads>
40+
<DefineConstants Condition="'$(WasmEnableThreads)' == 'true'">$(DefineConstants);FEATURE_WASM_MANAGED_THREADS</DefineConstants>
3841
</PropertyGroup>
3942
<PropertyGroup>
4043
<FeatureHardwareIntrinsics>true</FeatureHardwareIntrinsics>
@@ -347,6 +350,8 @@
347350
<Compile Include="System\Diagnostics\StackFrame.NativeAot.Wasm.cs" />
348351
<Compile Include="System\Diagnostics\StackFrame.NativeAot.Wasi.cs" Condition="'$(TargetsWasi)' == 'true'" />
349352
<Compile Include="System\Diagnostics\StackFrame.NativeAot.Browser.cs" Condition="'$(TargetsBrowser)' == 'true'" />
353+
<Compile Include="$(MonoBclSourcesRoot)\System\Threading\PortableThreadPool.Browser.Threads.Mono.cs" Condition="'$(TargetsBrowser)' == 'true' and '$(WasmEnableThreads)' == 'true'" />
354+
<Compile Include="$(MonoBclSourcesRoot)\System\Threading\ThreadPool.Browser.Threads.Mono.cs" Condition="'$(TargetsBrowser)' == 'true' and '$(WasmEnableThreads)' == 'true'" />
350355
</ItemGroup>
351356

352357
<!-- WASM-specific things. Keep in sync with Mono. -->

‎src/coreclr/nativeaot/System.Private.CoreLib/src/System/Threading/Monitor.NativeAot.cs‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,7 +143,9 @@ public static bool IsEntered(object obj)
143143

144144
#region Public Wait/Pulse methods
145145

146+
#if !FEATURE_WASM_MANAGED_THREADS
146147
[UnsupportedOSPlatform("browser")]
148+
#endif
147149
public static bool Wait(object obj, int millisecondsTimeout)
148150
{
149151
return GetCondition(obj).Wait(millisecondsTimeout, obj);

‎src/coreclr/runtime.proj‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,10 @@
9494
<_CoreClrBuildArg Include="-cmakeargs &quot;-DCLR_CMAKE_ESRP_CLIENT=$(DotNetEsrpToolPath)&quot;" />
9595
</ItemGroup>
9696

97+
<ItemGroup Condition="'$(WasmEnableThreads)' == 'true'">
98+
<_CoreClrBuildArg Include="-cmakeargs &quot;-DCMAKE_USE_PTHREADS=1&quot;" />
99+
</ItemGroup>
100+
97101
<PropertyGroup>
98102
<_CoreClrBuildScript Condition="$([MSBuild]::IsOsPlatform(Windows))">build-runtime.cmd</_CoreClrBuildScript>
99103
<_CoreClrBuildScript Condition="!$([MSBuild]::IsOsPlatform(Windows))">build-runtime.sh</_CoreClrBuildScript>

0 commit comments

Comments
 (0)