Skip to content

Commit 91e442b

Browse files
PathogenDavidPathogenDavid
committed
Reorganized native builds, native packages, and CI. Added Linux ARM64 and macOS x64 builds to CI.
* Moved native builds under `bin` folder, organized by corresponding .NET RID. * Renamed `ClangSharp.Pathogen.Runtime` to `ClangSharp.Pathogen.Native.<rid>` with a separate package per RID. * Moved Clang resources to dedicated `ClangSharp.Pathogen.ClangResources` package. * `ClangSharp.Pathogen` no longer directly depends on the native runtime package. * It's now expected consumers will reference the ones needed by their target platform. Biohazrd will reference `win-x64` and `linux-x64` by default and auomatically suggest other ones as needed. * If NuGet ever gets platform-specific depenednecy support we'll move towards that. * Version compatibility is now enforced by MSBuild targets in the packages rather than by NuGet. * Added workaround for dotnet/sourcelink#675 to avoid unecessary clones of LLVM. * ClangSharp.Pathogen now uses a release builds instead of debug. * Did general cleanup of CI workflow. * Added missing libClangSharp copyright info to native runtime package. * Added `LibClangSharpResolver.TryLoadExplicitly` for graceful testing of whether the native runtime can be loaded. * Fixed theoretical bug where the use of `DefaultDllImportSearchPathsAttribute` on libclang functions could mess with `LibClangSharpResolver`. * Fixed `create-llvm-vs-solution` not blocking `Directory.Build.*` files early enough. * Updated LLVM to fix CMP0116 warning spam from CMake. * Cleaned up readme a bit.
1 parent 8a4720d commit 91e442b

22 files changed

+550
-225
lines changed

‎.github/workflows/ClangSharp.Pathogen.yml‎

Lines changed: 158 additions & 60 deletions
Large diffs are not rendered by default.

‎.github/workflows/install-sccache.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@
4444
# Figure out the sccache directory paths and create it if necessary
4545
# We assume our working directory is the repo root.
4646
# The binary is placed in a subdirectory of its download hash to ensure we don't re-use an old version.
47-
sccache_root = os.path.join(os.getcwd(), "build-sccache")
47+
sccache_root = os.path.join(os.getcwd(), "bin", "tools", "sccache")
4848
sccache_cache_directory = os.path.join(sccache_root, "cache")
4949
sccache_cache_log_file_path = os.path.join(sccache_root, "sccache.log")
5050

‎.github/workflows/keep-cache-warm.yml‎

Lines changed: 26 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -25,19 +25,29 @@ jobs:
2525
fail-fast: false
2626
matrix:
2727
include:
28-
- os: windows-latest
29-
name: Windows
30-
lib-path: build/bin/libclang.dll
28+
- os: windows-2019
29+
name: Windows x64
30+
rid: win-x64
31+
libclang-subpath: bin/libclang.dll
3132
build-command: ./build-native.cmd
32-
artifact-name: LlvmBuildOutputs-Windows
3333
# We use Ubuntu 18.04 so that the binary is usable on systems using glibc 2.27 and later
3434
# If we want to support even older versions we should explore building libclang with https://github.com/wheybags/glibc_version_header
3535
- os: ubuntu-18.04
36-
name: Linux
37-
lib-path: build-linux/lib/libclang.so
38-
clang-resource-dir: build-linux/lib/clang/
36+
name: Linux x64
37+
rid: linux-x64
38+
libclang-subpath: lib/libclang.so
39+
build-command: ./build-native.sh
40+
- os: ubuntu-arm64-latest
41+
name: Linux ARM64
42+
rid: linux-arm64
43+
libclang-subpath: lib/libclang.so
44+
build-command: ./build-native.sh
45+
skip-python-install: true
46+
- os: macos-10.15
47+
name: MacOS x64
48+
rid: osx-x64
49+
libclang-subpath: lib/libclang.dylib
3950
build-command: ./build-native.sh
40-
artifact-name: LlvmBuildOutputs-Linux
4151
name: Warm cache - ${{matrix.name}}
4252
runs-on: ${{matrix.os}}
4353
steps:
@@ -49,6 +59,7 @@ jobs:
4959

5060
# ----------------------------------------------------------------------- Setup Python
5161
- name: Setup Python 3.8
62+
if: matrix.skip-python-install != true
5263
uses: actions/setup-python@v2
5364
with:
5465
python-version: '3.8'
@@ -63,22 +74,23 @@ jobs:
6374
- name: Install sccache
6475
id: sccache
6576
run: python .github/workflows/install-sccache.py
66-
77+
6778
# ----------------------------------------------------------------------- Warm LLVM build outputs
6879
- name: Load cached LLVM build outputs
6980
id: cached-llvm
7081
uses: PathogenDavid/cache@no-save-v2
7182
with:
72-
key: llvm-output-${{runner.os}}-${{steps.llvm.outputs.revision}}
83+
key: llvm-output-${{matrix.rid}}-${{steps.llvm.outputs.revision}}
7384
path: |
74-
${{matrix.lib-path}}
75-
${{matrix.clang-resource-dir}}
85+
bin/llvm/${{matrix.rid}}/${{matrix.libclang-subpath}}
86+
bin/llvm/${{matrix.rid}}/lib/clang/
87+
external/llvm-project/clang/LICENSE.TXT
7688
7789
# ----------------------------------------------------------------------- Warm sccache cache
7890
- name: Load LLVM sccache
7991
id: cached-sccache
8092
uses: PathogenDavid/cache@no-save-v2
8193
with:
8294
path: ${{steps.sccache.outputs.root-directory}}
83-
key: sccache-cache-${{runner.os}}-${{steps.llvm.outputs.revision}}
84-
restore-keys: sccache-cache-${{runner.os}}-
95+
key: sccache-cache-${{matrix.rid}}-${{steps.llvm.outputs.revision}}
96+
restore-keys: sccache-cache-${{matrix.rid}}-

‎.gitignore‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
11
.vs/
2-
/build*/
32
bin/
43
obj/
54
/packages/

‎ClangSharp.Pathogen.ClangResources/ClangSharp.Pathogen.ClangResources.csproj‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
<Project Sdk="Microsoft.NET.Sdk">
2+
<PropertyGroup>
3+
<TargetFramework>netstandard2.0</TargetFramework>
4+
<RuntimeIdentifier Condition="'$(RuntimeIdentifier)' == ''">$(NETCoreSdkRuntimeIdentifier)</RuntimeIdentifier>
5+
6+
<!-- Determine path to Clang resources -->
7+
<ClangResourceDirectoryPath>$(LlvmBuildOutputRoot)linux-x64/lib/clang/$(LlvmVersion)/</ClangResourceDirectoryPath>
8+
<!-- For local builds allow using non-linux-x64 resources -->
9+
<ClangResourceDirectoryPath Condition="'$(ContinuousIntegrationBuild)' != 'true' and !Exists('$(ClangResourceDirectoryPath)')">$(LlvmBuildOutputRoot)$(RuntimeIdentifier)/lib/clang/$(LlvmVersion)/</ClangResourceDirectoryPath>
10+
11+
<!-- Package Info -->
12+
<PackageLicenseFile>LICENSE.txt</PackageLicenseFile>
13+
<Description>This package exists primarily to support Biohazrd and is not intended for direct consumption.&#10;&#10;Clang's supporting resource files.</Description>
14+
<!-- This package only contains unmodified Clang files so we only list LLVM Team as the author/copyright holder. -->
15+
<Authors>LLVM Team</Authors>
16+
<!-- LLVM does not have a good canonical copyright string. They're supposed to put it in their source file as part of Apache 2.0 but they don't. -->
17+
<Copyright>LLVM copyright (c) 2003-2019 University of Illinois at Urbana-Champaign, LLVM Team.</Copyright>
18+
19+
<!-- This package is only for Clang's supporting resources, don't try to build/pack the .NET assembly or specify a framework dependency -->
20+
<IncludeBuildOutput>false</IncludeBuildOutput>
21+
<SuppressDependenciesWhenPacking>true</SuppressDependenciesWhenPacking>
22+
</PropertyGroup>
23+
<ItemGroup>
24+
<None Condition="Exists('$(ClangLicensePath)')" Include="$(ClangLicensePath)" Link="LICENSE.txt" Pack="true" PackagePath="LICENSE.txt" />
25+
<None Include="$(ClangResourceDirectoryPath)**/*" LinkBase="clang-resources" Condition="Exists('$(ClangResourceDirectoryPath)')">
26+
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
27+
<Pack>true</Pack>
28+
<!--
29+
We use MSBuild to copy the Clang resource directory to the build output of dependents using MSBuild instead of ContentFiles primarily to avoid having them pollute Solution Explorer:
30+
https://github.com/NuGet/Home/issues/4856
31+
-->
32+
<PackagePath>clang-resources/</PackagePath>
33+
</None>
34+
<None Update="ClangSharp.Pathogen.ClangResources.props">
35+
<Pack>true</Pack>
36+
<!--
37+
buildTransitive is required to ensure the resource directory is also copied to indirect dependents
38+
We put the props file in both build and buildTransitive for legacy compatibility even though we don't really expect legacy consumers. (IE: Visual Studio 2017)
39+
NuGet will only use the one from buildTransitive when it's buildTransitive-aware.
40+
https://github.com/NuGet/Home/wiki/Allow-package%2D-authors-to-define-build-assets-transitive-behavior
41+
-->
42+
<PackagePath>build;buildTransitive</PackagePath>
43+
</None>
44+
</ItemGroup>
45+
<!-- Error if any of the required files are missing -->
46+
<Target Name="_ClangResourceChecks" BeforeTargets="Build;Pack">
47+
<Error Text="Clang LICENSE.TXT is missing, ensure Git submodules are up-to-date." Condition="!Exists('$(ClangLicensePath)')" />
48+
<Error Text="Clang resource directory is missing." Condition="!Exists('$(ClangResourceDirectoryPath)')" />
49+
</Target>
50+
</Project>

ClangSharp.Pathogen.Runtime/ClangSharp.Pathogen.Runtime.props renamed to ClangSharp.Pathogen.ClangResources/ClangSharp.Pathogen.ClangResources.props

File renamed without changes.

‎ClangSharp.Pathogen.Native/ClangSharp.Pathogen.Native.csproj‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
<Project Sdk="Microsoft.NET.Sdk">
2+
<PropertyGroup>
3+
<TargetFramework>netstandard2.0</TargetFramework>
4+
<RuntimeIdentifier Condition="'$(RuntimeIdentifier)' == ''">$(NETCoreSdkRuntimeIdentifier)</RuntimeIdentifier>
5+
6+
<!-- ==============================================================================================================
7+
Paths to LLVM things
8+
=============================================================================================================== -->
9+
<LibClangSubPath Condition="$(RuntimeIdentifier.StartsWith('win-'))">bin/libclang.dll</LibClangSubPath>
10+
<LibClangSubPath Condition="$(RuntimeIdentifier.StartsWith('linux-'))">lib/libclang.so</LibClangSubPath>
11+
<LibClangSubPath Condition="$(RuntimeIdentifier.StartsWith('osx-'))">lib/libclang.dylib</LibClangSubPath>
12+
<LibClangPathogenFileName Condition="$(RuntimeIdentifier.StartsWith('win-'))">libclang-pathogen.dll</LibClangPathogenFileName>
13+
<LibClangPathogenFileName Condition="$(RuntimeIdentifier.StartsWith('linux-'))">libclang-pathogen.so</LibClangPathogenFileName>
14+
<LibClangPathogenFileName Condition="$(RuntimeIdentifier.StartsWith('osx-'))">libclang-pathogen.dylib</LibClangPathogenFileName>
15+
16+
<LibClangPath>$(LlvmBuildOutputRoot)$(RuntimeIdentifier)/$(LibClangSubPath)</LibClangPath>
17+
<PathogenExtensionsCppPath>$(LlvmSourceRoot)clang/tools/libclang/PathogenExtensions.cpp</PathogenExtensionsCppPath>
18+
19+
<!-- ==============================================================================================================
20+
Package Info
21+
=============================================================================================================== -->
22+
<PackageId>$(MSBuildProjectName).$(RuntimeIdentifier)</PackageId>
23+
<PackageLicenseFile>LICENSE.txt</PackageLicenseFile>
24+
<Description>Native runtime support package for ClangSharp.Pathogen on $(RuntimeIdentifier).&#10;&#10;This package exists primarily to support Biohazrd and should only be used as instructed.</Description>
25+
<!-- Microsoft is here because we embed components of libClangSharp -->
26+
<Authors>LLVM Team, Microsoft, David Maas, and Contributors</Authors>
27+
<!-- LLVM does not have a good canonical copyright string. They're supposed to put it in their source file as part of Apache 2.0 but they don't. -->
28+
<Copyright>Pathogen extensions copyright David Maas and Contributors. libClangSharp copyright © Microsoft and Contributors. LLVM copyright (c) 2003-2019 University of Illinois at Urbana-Champaign, LLVM Team.</Copyright>
29+
30+
<!-- This package is only for native runtime, don't try to build/pack the .NET assembly or specify a framework dependency -->
31+
<IncludeBuildOutput>false</IncludeBuildOutput>
32+
<SuppressDependenciesWhenPacking>true</SuppressDependenciesWhenPacking>
33+
</PropertyGroup>
34+
<ItemGroup>
35+
<None Condition="Exists('$(PathogenExtensionsCppPath)')" Include="$(PathogenExtensionsCppPath)" Link="PathogenExtensions.cpp" />
36+
<None Condition="Exists('$(ClangLicensePath)')" Include="$(ClangLicensePath)" Link="LICENSE.txt" Pack="true" PackagePath="LICENSE.txt" />
37+
<None Include="$(LibClangPath)" Link="$(LibClangPathogenFileName)" Condition="Exists('$(LibClangPath)')">
38+
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
39+
<Pack>true</Pack>
40+
<PackagePath>runtimes/$(RuntimeIdentifier)/native/$(LibClangPathogenFileName)</PackagePath>
41+
</None>
42+
</ItemGroup>
43+
<!-- Error if native runtime needs to be built or llvm-project is missing -->
44+
<Target Name="_NativeRuntimeChecks" BeforeTargets="GetCopyToOutputDirectoryItems;Build;GenerateNuspec;Pack">
45+
<!-- PathogenExtensions.cpp is only used to check if LLVM is cloned and it's our fork. Don't check for it on CI where we might not actually clone LLVM. -->
46+
<Error Text="PathogenExtensions.cpp is missing, ensure Git submodules are up-to-date." Condition="!Exists('$(PathogenExtensionsCppPath)') and '$(ContinuousIntegrationBuild)' != 'true'" />
47+
<Error Text="Clang LICENSE.TXT is missing, ensure Git submodules are up-to-date." Condition="!Exists('$(ClangLicensePath)')" />
48+
<Error Text="Could not automatically determine LibClangSubPath based on runtime identifier '$(RuntimeIdentifier)'" Condition="'$(LibClangSubPath)' == ''" />
49+
<Error Text="Could not automatically determine LibClangPathogenFileName based on runtime identifier '$(RuntimeIdentifier)'" Condition="'$(LibClangPathogenFileName)' == ''" />
50+
<Error Text="Could not locate native runtime for '$(RuntimeIdentifier)' at '$(LibClangPath)', did you run build-native.cmd/sh?" Condition="!Exists('$(LibClangPath)')" />
51+
</Target>
52+
</Project>

‎ClangSharp.Pathogen.Runtime/ClangSharp.Pathogen.Runtime.csproj‎

Lines changed: 0 additions & 86 deletions
This file was deleted.

‎ClangSharp.Pathogen.sln‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,9 @@ VisualStudioVersion = 17.0.31612.314
55
MinimumVisualStudioVersion = 10.0.40219.1
66
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "ClangSharp.Pathogen", "ClangSharp.Pathogen\ClangSharp.Pathogen.csproj", "{ADCA9D07-0CAB-4D1F-92DD-69189D520EFB}"
77
EndProject
8-
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "ClangSharp.Pathogen.Runtime", "ClangSharp.Pathogen.Runtime\ClangSharp.Pathogen.Runtime.csproj", "{3A053B1B-DB74-48A3-994C-16C48AA2378A}"
8+
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "ClangSharp.Pathogen.Native", "ClangSharp.Pathogen.Native\ClangSharp.Pathogen.Native.csproj", "{3A053B1B-DB74-48A3-994C-16C48AA2378A}"
9+
EndProject
10+
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ClangSharp.Pathogen.ClangResources", "ClangSharp.Pathogen.ClangResources\ClangSharp.Pathogen.ClangResources.csproj", "{98F8BC0B-125E-4CC0-BB5B-0A7E23E70F5D}"
911
EndProject
1012
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution Items", "{D91EF5C2-A8C3-450F-9369-2A6E1BA256C4}"
1113
ProjectSection(SolutionItems) = preProject
@@ -35,6 +37,10 @@ Global
3537
{3A053B1B-DB74-48A3-994C-16C48AA2378A}.Debug|Any CPU.Build.0 = Debug|Any CPU
3638
{3A053B1B-DB74-48A3-994C-16C48AA2378A}.Release|Any CPU.ActiveCfg = Release|Any CPU
3739
{3A053B1B-DB74-48A3-994C-16C48AA2378A}.Release|Any CPU.Build.0 = Release|Any CPU
40+
{98F8BC0B-125E-4CC0-BB5B-0A7E23E70F5D}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
41+
{98F8BC0B-125E-4CC0-BB5B-0A7E23E70F5D}.Debug|Any CPU.Build.0 = Debug|Any CPU
42+
{98F8BC0B-125E-4CC0-BB5B-0A7E23E70F5D}.Release|Any CPU.ActiveCfg = Release|Any CPU
43+
{98F8BC0B-125E-4CC0-BB5B-0A7E23E70F5D}.Release|Any CPU.Build.0 = Release|Any CPU
3844
EndGlobalSection
3945
GlobalSection(SolutionProperties) = preSolution
4046
HideSolutionNode = FALSE

0 commit comments

Comments
 (0)