Merge pull request swiftlang#27869 from apple/compnerd-monorepo-windows

Update WindowsBuild.md

Author: compnerd

docs/WindowsBuild.md

@@ -8,20 +8,10 @@ Visual Studio 2017 or newer is needed to build swift on Windows.
   Development|Universal Windows App Development|Windows SDK" in your
   installation.
 
-## 2. Clone the repositories
-1. Configure git to work with Unix file endings (ie `git config --global core.autocrlf false`)
-1. Create a folder to contain all the Swift repositories
-1. Clone `apple/swift-cmark` into a folder named `cmark`
-1. Clone `apple/swift-clang` into a folder named `clang`
-1. Clone `apple/swift-llvm` into a folder named `llvm`
-1. Clone `apple/swift-compiler-rt` into a folder named `compiler-rt`
-1. Clone `apple/swift` into a folder named `swift`
-1. Clone `apple/swift-corelibs-libdispatch` into a folder named `swift-corelibs-libdispatch`
-1. Clone `apple/swift-corelibs-foundation` into a folder named `swift-corelibs-foundation`
-1. Clone `apple/swift-corelibs-xctest` into a folder name `swift-corelibs-xctest`
-1. Clone `apple/swift-lldb` into a folder named `lldb`
-1. Clone `apple/swift-llbuild` into a folder named `llbuild`
-1. Clone `apple/swift-package-manager` info a folder named `swift-package-manager`
+## 1. Clone the repositories
+1. Clone `apple/llvm-project` into a directory for the toolchain
+1. Clone `apple/swift-cmark`, `apple/swift`, `apple/swift-corelibs-libdispatch`, `apple/swift-corelibs-foundation`, `apple/swift-corelibs-xctest`, `apple/swift-llbuild`, `apple/swift-package-manager` into the toolchain directory
+1. Clone `compnerd/windows-swift` as a peer of the toolchain directory
 
 - Currently, other repositories in the Swift project have not been tested and may not be supported.
 
@@ -33,20 +23,18 @@ subst S: <path to sources>
 
 ```cmd
 S:
-git clone https://github.com/apple/swift-cmark cmark
-git clone https://github.com/apple/swift-clang clang
-git clone https://github.com/apple/swift-llvm llvm
-git clone https://github.com/apple/swift-compiler-rt compiler-rt
-git clone -c core.autocrlf=input -c core.symlinks=true https://github.com/apple/swift
-git clone https://github.com/apple/swift-corelibs-libdispatch
-git clone https://github.com/apple/swift-corelibs-foundation
-git clone https://github.com/apple/swift-corelibs-xctest
-git clone https://github.com/apple/swift-lldb lldb
-git clone https://github.com/apple/swift-llbuild llbuild
-git clone -c core.autocrlf=input https://github.com/apple/swift-package-manager
-```
-
-## 3. Acquire ICU, SQLite3, curl, libxml2
+git clone https://github.com/apple/llvm-project --branch swift/master toolchain
+git clone -c core.autocrlf=input -c core.symlinks=true https://github.com/apple/swift toolchain/swift
+git clone https://github.com/apple/swift-cmark toolchain/cmark
+git clone https://github.com/apple/swift-corelibs-libdispatch toolchain/swift-corelibs-libdispatch
+git clone https://github.com/apple/swift-corelibs-foundation toolchain/swift-corelibs-foundation
+git clone https://github.com/apple/swift-corelibs-xctest toolchain/swift-corelibs-xctest
+git clone https://github.com/apple/swift-llbuild toolchain/llbuild
+git clone -c core.autocrlf=input https://github.com/apple/swift-package-manager toolchain/swift-package-manager
+git clone https://github.com/compnerd/windows-swift windows-swift
+```
+
+## 1. Acquire ICU, SQLite3, curl, libxml2
 1. Go to https://dev.azure.com/compnerd/windows-swift and scroll down to "Dependencies" where you'll see bots (hopefully green) for icu, SQLite, curl, and libxml2. Download each of the zip files and copy their contents into S:/Library. The directory structure should resemble:
 
 ```
@@ -58,7 +46,7 @@ S:/Library/
           /zlib-1.2.11/usr
 ```
 
-## 4. Get ready
+## 1. Get ready
 - From within a **NATIVE developer** command prompt (not PowerShell nor cmd, but the [Visual Studio Developer Command Prompt](https://msdn.microsoft.com/en-us/library/f35ctcxw.aspx)), execute the following command if you have an x64 PC (The Native Developer command prompt is situated at "C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Visual Studio 2019\Visual Studio Tools\VC\x64 Native Tools Command Prompt for VS 2019.lnk").
 
 Run this as administrator the first time, for setting up the symlinks below. 
@@ -87,114 +75,31 @@ mklink "%VCToolsInstallDir%\include\visualc.apinotes" S:\swift\stdlib\public\Pla
 
 Warning: Creating the above links usually requires administrator privileges. The quick and easy way to do this is to open a second developer prompt by right clicking whatever shortcut you used to open the first one, choosing Run As Administrator, and pasting the above commands into the resulting window. You can then close the privileged prompt; this is the only step which requires elevation.
 
-## 5. Build LLVM/Clang
-- This must be done from within a developer command prompt. Make sure that the build
-  type for LLVM/Clang is compatible with the build type for Swift. That is,
-  either build everything `Debug` or some variant of `Release` (e.g. `Release`,
-  `RelWithDebInfo`).
-```cmd
-md "S:\b\llvm"
-cd "S:\b\llvm"
-cmake -G Ninja^
- -DCMAKE_BUILD_TYPE=Release^
- -DCMAKE_C_COMPILER=cl^
- -DCMAKE_CXX_COMPILER=cl^
- -DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-windows-msvc^
- -DLLVM_ENABLE_ASSERTIONS=ON^
- -DLLVM_ENABLE_PDB=YES^
- -DLLVM_ENABLE_PROJECTS=clang^
- -DLLVM_TARGETS_TO_BUILD="AArch64;ARM;X86"^
- S:/llvm
-ninja
-```
-
-- Update your path to include the LLVM tools.
-
-```cmd
-path S:\b\llvm\bin;%PATH%
-```
-## 6. Build CMark
-- This must be done from within a developer command prompt.
-
-```cmd
-md "S:\b\cmark"
-cd "S:\b\cmark"
-cmake -G Ninja^
-  -DCMAKE_BUILD_TYPE=RelWithDebInfo^
-  -DCMAKE_C_COMPILER=cl^
-  -DCMAKE_CXX_COMPILER=cl^
-  S:\cmark
-ninja
-```
-
-## 7. Build Swift
-- This must be done from within a developer command prompt
-- Note that Visual Studio vends a 32-bit python 2.7 installation in `C:\Python27` and a 64-bit python in `C:\Python27amd64`.  You may use either one based on your installation.
-
-```cmd
-md "S:\b\swift"
-cd "S:\b\swift"
-cmake -G Ninja^
- -DCMAKE_BUILD_TYPE=RelWithDebInfo^
- -DCMAKE_C_COMPILER=cl^
- -DCMAKE_CXX_COMPILER=cl^
- -DCMAKE_EXE_LINKER_FLAGS:STRING="/INCREMENTAL:NO"^
- -DCMAKE_SHARED_LINKER_FLAGS:STRING="/INCREMENTAL:NO"^
- -DSWIFT_INCLUDE_DOCS=OFF^
- -DSWIFT_PATH_TO_CMARK_SOURCE="S:\cmark"^
- -DSWIFT_PATH_TO_CMARK_BUILD="S:\b\cmark"^
- -DLLVM_DIR=S:\b\llvm\lib\cmake\llvm^
- -DClang_DIR=S:\b\llvm\lib\cmake\clang^
- -DSWIFT_PATH_TO_LIBDISPATCH_SOURCE="S:\swift-corelibs-libdispatch"^
- -DSWIFT_WINDOWS_x86_64_ICU_UC_INCLUDE="S:/Library/icu-64/usr/include"^
- -DSWIFT_WINDOWS_x86_64_ICU_UC="S:/Library/icu-64/usr/lib/icuuc64.lib"^
- -DSWIFT_WINDOWS_x86_64_ICU_I18N_INCLUDE="S:/Library/icu-64/usr/include"^
- -DSWIFT_WINDOWS_x86_64_ICU_I18N="S:/Library/icu-64/usr/lib/icuin64.lib"^
- -DCMAKE_INSTALL_PREFIX="C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr"^
- -DPYTHON_EXECUTABLE=C:\Python27\python.exe^
- S:\swift
-ninja
-```
-
-- To create a Visual Studio project, you'll need to change the generator and,
-  if you have a 64 bit processor, specify the generator platform. Note that you
-  may get multiple build errors compiling the `swift` project due to an MSBuild
-  limitation that file paths cannot exceed 260 characters. These can be
-  ignored, as they occur after the build when writing the last build status to
-  a file.
+## 1. Build toolchain
+- This must be done from within a developer command prompt. Make sure that the build type for LLVM/Clang is compatible with the build type for Swift. That is, either build everything `Debug` or some variant of `Release` (e.g. `Release`, `RelWithDebInfo`).
 
 ```cmd
-cmake -G "Visual Studio 2017" -A x64 -T "host=x64"^ ...
+md "S:\b\toolchain"
+cmake -B "S:\b\toolchain" -G Ninja -C S:\windows-swift\cmake\caches\Windows-x86_64.cmake -C S:\windows-swift\cmake\caches\org.compnerd.dt.cmake -DLLVM_ENABLE_ASSERTIONS=YES -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;cmark;swift;lldb" -DLLVM_EXTERNAL_PROJECTS="cmark;swift" -DSWIFT_PATH_TO_LIBDISPATCH_SOURCE=S:\llvm-project\swift-corelibs-libdispatch -DLLVM_ENABLE_PDB=YES -DLLDB_DISABLE_PYTHON=YES -DSWIFT_WINDOWS_x86_64_ICU_UC_INCLUDE="S:/Library/icu-64/usr/include" -DSWIFT_WINDOWS_x86_64_ICU_UC="S:/Library/icu-64/usr/lib/icuuc64.lib" -DSWIFT_WINDOWS_x86_64_ICU_I18N_INCLUDE="S:/Library/icu-64/usr/include" -DSWIFT_WINDOWS_x86_64_ICU_I18N="S:/Library/icu-64/usr/lib/icuin64.lib" -DCMAKE_INSTALL_PREFIX="C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr" -DPYTHON_EXECUTABLE=C:\Python27\python.exe
+ninja -C S:\b\toolchain
 ```
 
-## 8. Build lldb
-- This must be done from within a developer command prompt.
+- Update your path to include the toolchain.
 
 ```cmd
-md "S:\b\lldb"
-cd "S:\b\lldb"
-cmake -G Ninja^
-  -DLLVM_DIR="S:/b/llvm/lib/cmake/llvm"^
-  -DClang_DIR="S:/b/llvm/lib/cmake/clang"^
-  -DSwift_DIR="S:/b/swift/lib/cmake/swift"^
-  -DCMAKE_BUILD_TYPE=RelWithDebInfo^
-  -DLLDB_USE_STATIC_BINDINGS=YES^
-  -DLLVM_ENABLE_ASSERTIONS=ON^
-  -DPYTHON_HOME="%ProgramFiles(x86)%\Microsoft Visual Studio\Shared\Python37_64"^
-  S:\lldb
-ninja
+path S:\b\toolchain\bin;%PATH%
 ```
 
-## 9. Running tests on Windows
+## 1. Running tests on Windows
 
 Running the testsuite on Windows has additional external dependencies.
 
 ```cmd
-path S:\Library\icu-64\usr\bin;S:\b\swift\bin;S:\b\swift\libdispatch-prefix\bin;%PATH%;%ProgramFiles%\Git\usr\bin
-ninja -C S:\b\swift check-swift
+path S:\Library\icu-64\usr\bin;S:\b\toolchain\bin;S:\b\toolchain\libdispatch-prefix\bin;%PATH%;%ProgramFiles%\Git\usr\bin
+ninja -C S:\b\toolchain check-swift
 ```
 
-## 10. Build swift-corelibs-libdispatch
+## 1. Build swift-corelibs-libdispatch
 
 ```cmd
 md "S:\b\libdispatch"
@@ -203,10 +108,10 @@ cmake -G Ninja^
   -DCMAKE_BUILD_TYPE=RelWithDebInfo^
   -DCMAKE_C_COMPILER=clang-cl^
   -DCMAKE_CXX_COMPILER=clang-cl^
-  -DCMAKE_SWIFT_COMPILER=S:\b\swift\bin\swiftc.exe^
+  -DCMAKE_SWIFT_COMPILER=S:\b\toolchain\bin\swiftc.exe^
   -DENABLE_SWIFT=ON^
   -DENABLE_TESTING=OFF^
-  S:\swift-corelibs-libdispatch
+  S:\toolchain\swift-corelibs-libdispatch
 ninja
 ```
 
@@ -215,21 +120,21 @@ ninja
 path S:\b\libdispatch;S:\b\libdispatch\src;%PATH%
 ```
 
-## 11. Build swift-corelibs-foundation
+## 1. Build swift-corelibs-foundation
 
 ```cmd
 md "S:\b\foundation"
 cd "S:\b\foundation
 cmake -G Ninja^
   -DCMAKE_BUILD_TYPE=RelWithDebInfo^
   -DCMAKE_C_COMPILER=clang-cl^
-  -DCMAKE_SWIFT_COMPILER=S:\b\swift\bin\swiftc.exe^
-  -DCURL_LIBRARY="S:/curl/builds/libcurl-vc15-x64-release-static-ipv6-sspi-winssl/lib/libcurl_a.lib"^
-  -DCURL_INCLUDE_DIR="S:/curl/builds/libcurl-vc15-x64-release-static-ipv6-sspi-winssl/include"^
+  -DCMAKE_SWIFT_COMPILER=S:\b\toolchain\bin\swiftc.exe^
+  -DCURL_LIBRARY="S:/Library/libcurl-development/usr/lib/libcurl.lib"^
+  -DCURL_INCLUDE_DIR="S:/Library/libcurl-development/usr/include"^
   -DENABLE_TESTING=NO^
   -DICU_ROOT="S:/Library/icu-64"^
-  -DLIBXML2_LIBRARY="S:/libxml2/win32/bin.msvc/libxml2_a.lib"^
-  -DLIBXML2_INCLUDE_DIR="S:/libxml2/include"^
+  -DLIBXML2_LIBRARY="S:/Library/libxml2-development/usr/lib/libxml2.lib"^
+  -DLIBXML2_INCLUDE_DIR="S:/Library/libxml2-development/usr/include"^
   -DFOUNDATION_PATH_TO_LIBDISPATCH_SOURCE=S:\swift-corelibs-libdispatch^
   -DFOUNDATION_PATH_TO_LIBDISPATCH_BUILD=S:\b\libdispatch^
    S:\swift-corelibs-foundation
@@ -241,15 +146,15 @@ ninja
 path S:\b\foundation;%PATH%
 ```
 
-## 12. Build swift-corelibs-xctest
+## 1. Build swift-corelibs-xctest
 
 ```cmd
 md "S:\b\xctest"
 cd "S:\b\xctest"
 cmake -G Ninja^
   -DBUILD_SHARED_LIBS=YES^
   -DCMAKE_BUILD_TYPE=RelWithDebInfo^
-  -DCMAKE_SWIFT_COMPILER=S:\b\swift\bin\swiftc.exe^
+  -DCMAKE_SWIFT_COMPILER=S:\b\toolchain\bin\swiftc.exe^
   -DXCTEST_PATH_TO_FOUNDATION_BUILD=S:\b\foundation^
   -DXCTEST_PATH_TO_LIBDISPATCH_SOURCE=S:\swift-corelibs-libdispatch^
   -DXCTEST_PATH_TO_LIBDISPATCH_BUILD=S:\b\libdispatch^
@@ -264,41 +169,41 @@ ninja
 path S:\b\xctest;%PATH%
 ```
 
-## 13. Test XCTest
+## 1. Test XCTest
 
 ```cmd
 ninja -C S:\b\xctest check-xctest
 ```
 
-## 14. Rebuild Foundation
+## 1. Rebuild Foundation
 
 ```cmd
 cd "S:\b\foundation
 cmake -G Ninja^
   -DCMAKE_BUILD_TYPE=RelWithDebInfo^
   -DCMAKE_C_COMPILER=clang-cl^
-  -DCMAKE_SWIFT_COMPILER=S:\b\swift\bin\swiftc.exe^
-  -DCURL_LIBRARY="S:/curl/builds/libcurl-vc15-x64-release-static-ipv6-sspi-winssl/lib/libcurl_a.lib"^
-  -DCURL_INCLUDE_DIR="S:/curl/builds/libcurl-vc15-x64-release-static-ipv6-sspi-winssl/include"^
+  -DCMAKE_SWIFT_COMPILER=S:\b\toolchain\bin\swiftc.exe^
+  -DCURL_LIBRARY="S:/Library/libcurl-development/usr/lib/libcurl.lib"^
+  -DCURL_INCLUDE_DIR="S:/Library/libcurl-development/usr/include"^
   -DENABLE_TESTING=YES^
   -DICU_ROOT="S:/Library/icu-64"^
-  -DLIBXML2_LIBRARY="S:/libxml2/win32/bin.msvc/libxml2_a.lib"^
-  -DLIBXML2_INCLUDE_DIR="S:/libxml2/include"^
+  -DLIBXML2_LIBRARY="S:/Library/libxml2-development/usr/lib/libxml2.lib"^
+  -DLIBXML2_INCLUDE_DIR="S:/Library/libxml2-development/usr/include"^
   -DFOUNDATION_PATH_TO_LIBDISPATCH_SOURCE=S:\swift-corelibs-libdispatch^
   -DFOUNDATION_PATH_TO_LIBDISPATCH_BUILD=S:\b\libdispatch^
   -DFOUNDATION_PATH_TO_XCTEST_BUILD=S:\b\xctest^
    S:\swift-corelibs-foundation
 ninja
 ```
 
-## 15. Test Foundation
+## 1. Test Foundation
 
 ```cmd
 cmake --build S:\b\foundation
 ninja -C S:\b\foundation test 
 ```
 
-## 16. Build llbuild
+## 1. Build llbuild
 
 ```cmd
 md S:\b\llbuild
@@ -311,8 +216,8 @@ cmake -G Ninja^
   -DFOUNDATION_BUILD_DIR=S:\b\foundation^
   -DLIBDISPATCH_BUILD_DIR=S:\b\libdispatch^
   -DLIBDISPATCH_SOURCE_DIR=S:\swift-corelibs-libdispatch^
-  -DSQLite3_INCLUDE_DIR=S:\sqlite-amalgamation-3270200^
-  -DSQLite3_LIBRARY=S:\b\sqlite\sqlite3.lib^
+  -DSQLite3_INCLUDE_DIR=S:\Library\sqlite-3.28.0\usr\include^
+  -DSQLite3_LIBRARY=S:\Library\sqlite-3.28.0\usr\lib\sqlite3.lib^
   -DLLBUILD_SUPPORT_BINDINGS=Swift^
   S:\llbuild
 ninja
@@ -323,15 +228,15 @@ ninja
 path S:\b\llbuild\bin;%PATH%
 ```
 
-## 17. Build swift-package-manager
+## 1. Build swift-package-manager
 
 ```cmd
 md S:\b\spm
 cd S:\b\spm
 C:\Python27\python.exe S:\swift-package-manager\Utilities\bootstrap --foundation S:\b\foundation --libdispatch-build-dir S:\b\libdispatch --libdispatch-source-dir S:\swift-corelibs-libdispatch --llbuild-build-dir S:\b\llbuild --llbuild-source-dir S:\llbuild --sqlite-build-dir S:\b\sqlite --sqlite-source-dir S:\sqlite-amalgamation-3270200
 ```
 
-## 18. Install Swift on Windows
+## 1. Install Swift on Windows
 
 - Run ninja install: