From e4efbeb02347e7a464b4809378fa851880427751 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sat, 10 Aug 2024 08:16:49 -0400 Subject: [PATCH 01/26] DRAFT: Swiftly proxies Add a design proposal for the new swiftly proxy system --- DESIGN.md | 41 ++++++++++++++++++++++------------------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index dd0319aa..2aa69926 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -60,7 +60,7 @@ A simple setup for managing the toolchains could look like this: The toolchains (i.e. the contents of a given Swift download tarball) would be contained in the toolchains directory, each named according to the major/minor/patch version. `config.json` would contain any required metadata (e.g. the latest Swift version, which toolchain is selected, etc.). If pulling in Foundation to use `JSONEncoder`/`JSONDecoder` (or some other JSON tool) would be a problem, we could also use something simpler. -The `~/.local/bin` directory would include symlinks pointing to the `bin` directory of the "active" toolchain, if any. +The `~/.local/bin` directory would include symlinks pointing to swiftly itself. When these binaries are run swiftly proxies them to the requested toolchain, or a default. This is all very similar to how rustup does things, but I figure there's no need to reinvent the wheel here. @@ -78,7 +78,7 @@ The contents of `~/Library/Application Support/swiftly` would look like this: – env ``` -Instead of downloading tarballs containing the toolchains and storing them directly in `~/.local/share/swiftly/toolchains`, we instead install Swift toolchains to `~/Library/Developer/Toolchains` via the `.pkg` files provided for download at swift.org. To select a toolchain for use, we update the symlinks at `~/Library/Application Support/swiftly/bin` to point to the desired toolchain in `~/Library/Developer/Toolchains`. In the env file, we’ll contain a line that looks like `export PATH="$HOME/Library/Application Support/swiftly:$PATH"`, so the version of swift being used will automatically always be from the active toolchain. `config.json` will contain version information about the selected toolchain as well as its actual location on disk. +Instead of downloading tarballs containing the toolchains and storing them directly in `~/.local/share/swiftly/toolchains`, we instead install Swift toolchains to `~/Library/Developer/Toolchains` via the `.pkg` files provided for download at swift.org. In the env file, we’ll add a line that looks like `export PATH="$HOME/Library/Application Support/swiftly:$PATH"`, so that swiftly can proxy toolchain commands to the requested toolchain, or default. `config.json` will contain version information about the selected toolchain as well as its actual location on disk. This scheme works for ensuring the version of Swift used on the command line can be controlled, but it doesn’t affect the active toolchain used by Xcode, which uses its own mechanisms for that. Xcode, if it is installed, can find the toolchains installed by swiftly. @@ -178,7 +178,7 @@ To list all the versions of swift installed on your system #### use -“Using” a toolchain sets it as the active toolchain, meaning it will be the one found via $PATH and invoked via `swift` commands executed in the shell. Only a single toolchain can be used at a given time. Using a toolchain doesn’t uninstall anything; it only updates symlinks so that the requested toolchain can be found by the shell. +“Using” a toolchain sets it as the default toolchain, meaning it will be the default one that is used when running toolchain commands from the shell. Only a single toolchain can be the default at a given time and location. Using a toolchain doesn’t uninstall anything; it only updates the configuration. To use the toolchain associated with the most up-to-date Swift version, the “latest” version can be specified: @@ -208,6 +208,10 @@ To use the latest installed main snapshot, leave off the date: `swiftly use main-snapshot` +The use subcommand also supports `.swift-version` files. If version file is present in the current working directory, or an ancestory directory, then swiftly will update that file with the new version to use. This can be a useful feature for a team to share and align on toolchain versions with git. As a special case, if swiftly could not find a version file, but it could find a Package.swift file it will create a new version file for you in the package and set that to the requested toolchain version. + +Note: The `.swift-version` file mechanisms can be overridden using the `--global-default` flag so that your swiftly installation's default toolchain can be set explicitly. + #### update Update replaces a given toolchain with a later version of that toolchain. For a stable release, this means updating to a later patch version. For snapshots, this means updating to the most recently available snapshot. @@ -266,6 +270,18 @@ This command checks to see if there are new versions of `swiftly` itself and upg `swiftly self-update` +### Proxy command invocations to a requested toolchain + +Swiftly will create a set of symbolic links in its SWIFTLY_BIN_DIR during installation that point to the swiftly binary itself for each of the common toolchain commands, such as swift, swiftc, clang, etc. This mechanism will allows swiftly to proxy those command invocations to a requested toolchain. A toolchain can be requested in these ways in order of precedence: + +* Special toolchain selectors among the regular tool command-line arguments (e.g. `swift build +5.10.1`) with the special '+' prefix +* The presence of a .swift-version file in the current working directory, or ancestor directory, with the required toolchain version +* The swiftly default (in-use) toolchain set in the config.json by `swiftly install` or `swiftly use` commands + +In the first two cases, if there is no matching toolchain installed, swiftly will attempt to automatically install the requested toolchain and use it if the installation succeeeds. + +Note: If swiftly automatically installs a toolchain during proxying and that toolchain requires post installation steps then the proxy will abort with those post installation instructions. + ## Detailed Design Swiftly itself will be a SPM project consisting of several executable products, one per supported platform, and all of these will share the core module that handles argument parsing, printing help information, and dispatching commands. Each platform’s executable will be built to statically link the stdlib so that they can be run without having installed Swift first. @@ -457,15 +473,7 @@ https://download.swift.org/swift-5.5.1-release/ubuntu1604/swift-5.5.1-RELEASE/sw $ tar -xf --directory ~/.local/share/swiftly/toolchains ``` -It also updates `config.json` to include this toolchain as the latest for the provided version. If installing a new patch release toolchain, the now-outdated one can be deleted (e.g. `5.5.0` can be deleted when `5.5.1` is installed). - -Finally, the use implementation executes the following to update the link: - -``` -$ ln -s ~/.local/share/swiftly/toolchains//usr/bin/swift ~/.local/bin/swift -``` - -It also updates `config.json` to include this version as the currently selected one. +It also updates `config.json` to include this toolchain as the latest for the provided version. If installing a new patch release toolchain, the now-outdated one can be deleted (e.g. `5.5.0` can be deleted when `5.5.1` is installed). The `config.json` is updated to include this version as the currently selected (default) one. ### Implementation Sketch - macOS @@ -481,18 +489,13 @@ https://download.swift.org/swift--RELEASE/xcode/swift--RELEASE `config.json` is then updated to include this toolchain as the latest for the provided version. -Finally, the use implementation executes the following to update the link: - -``` -$ ln -s ~/Library/Developer/Toolchains/ ~/.swiftly/active-toolchain -``` - -It also updates `config.json` to include this version as the currently selected one. +It also updates `config.json` to include this version as the currently selected (default) one. ### `config.json` Schema ``` { + "version": "", "platform": { "namePretty": , "fullName": , From 3b5c4044bb6f8d4c867f86ede5acc76b5d448a91 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sat, 10 Aug 2024 08:22:13 -0400 Subject: [PATCH 02/26] Add a swiftly install workflow where the version comes from the .swift-version file --- DESIGN.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/DESIGN.md b/DESIGN.md index 2aa69926..c0b4e595 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -138,6 +138,14 @@ Installing a specific snapshot from a swift version development branch `swiftly install 5.5-snapshot-2022-1-28` +##### Installing the version from the `.swift-version` file + +A package could have a swift version file that specifies the recommended toolchain version. A swiftly install with no version will search for a version file and install that version. + +`swiftly install` + +If no swift version file can be found then the installation fails indicating that it couldn't fine the file. + #### uninstall Uninstalling versions of Swift should be in a similar form to install. Uninstalling a toolchain that is currently “in use” (see the “use” command section below) will cause swiftly to use the latest Swift release toolchain that is installed. If none are, the latest snapshot will be used. If no snapshots are installed either, then a message will be printed indicating that all Swift versions are uninstalled. From 956256ffe9bec0f684a3613c134debc8b93c53d5 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Tue, 20 Aug 2024 12:15:47 -0400 Subject: [PATCH 03/26] update design to make auto-installation an error instead --- DESIGN.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index c0b4e595..843ad2e4 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -286,9 +286,7 @@ Swiftly will create a set of symbolic links in its SWIFTLY_BIN_DIR during instal * The presence of a .swift-version file in the current working directory, or ancestor directory, with the required toolchain version * The swiftly default (in-use) toolchain set in the config.json by `swiftly install` or `swiftly use` commands -In the first two cases, if there is no matching toolchain installed, swiftly will attempt to automatically install the requested toolchain and use it if the installation succeeeds. - -Note: If swiftly automatically installs a toolchain during proxying and that toolchain requires post installation steps then the proxy will abort with those post installation instructions. +If swiftly cannot find an installed toolchain that matches the request then it fails with an error and instructions how to use `swiftly install` to fulfill the request. ## Detailed Design From 375df29f044160bf1cd04a93befb0046c3743eab Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Tue, 20 Aug 2024 15:15:27 -0400 Subject: [PATCH 04/26] provide a mechanism to find the currently in-use toolchain physical location clarify the boundaries of the swiftly toolchain abstraction and elaborate on how to work around them --- DESIGN.md | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index 843ad2e4..2ff4bf7b 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -4,21 +4,28 @@ This document contains the high level design of swiftly. Not all features have b ## Index +- [Swiftly's purpose](#swiftlys-purpose) +- [Installation of swiftly](#installation-of-switly) - [Linux](#linux) - - [Installation of swiftly](#installation-of-swiftly) - [Installation of a Swift toolchain](#installation-of-a-swift-toolchain) - [macOS](#macos) - - [Installation of swiftly](#installation-of-swiftly-1) - [Installation of a Swift toolchain](#installation-of-a-swift-toolchain-1) - [Interface](#interface) - [Toolchain names and versions](#toolchain-names-and-versions) - [Commands](#commands) + - [Toolchain selection](#toolchain-selection) - [Detailed design](#detailed-design) - [Implementation sketch - Core](#implementation-sketch---core) - [Implementation sketch - Ubuntu 20.04](#implementation-sketch---ubuntu-2004) - [Implementation sketch - macOS](#implementation-sketch---macos) - [`config.json` schema](#configjson-schema) +## Swiftly's purpose + +Swiftly helps you to easily install different Swift toolchains locally on your account. It also provides a single path where you can run the tools in the currently selected toolchain. Toolchain selection is [configurable](#toolchain-selection) using different mechanisms. + +Note that swiftly is *not* a virtual toolchain in itself since there are cases where it cannot behave as a self-contained Swift toolchain. For example, there can be external dependencies on specific files, such as headers or libraries, far too many and variable between toolchain versions to be managed by swiftly. Also, for long-lived processes, there is no way to gracefully restart them without help from the client. + ## Installation of swiftly The installation of swiftly is divided into two phases: delivery and initialization. Delivery of the swiftly binary can be accomplished using different methods: @@ -278,15 +285,25 @@ This command checks to see if there are new versions of `swiftly` itself and upg `swiftly self-update` -### Proxy command invocations to a requested toolchain +### Toolchain selection -Swiftly will create a set of symbolic links in its SWIFTLY_BIN_DIR during installation that point to the swiftly binary itself for each of the common toolchain commands, such as swift, swiftc, clang, etc. This mechanism will allows swiftly to proxy those command invocations to a requested toolchain. A toolchain can be requested in these ways in order of precedence: +Swiftly will create a set of symbolic links in its SWIFTLY_BIN_DIR during installation that point to the swiftly binary itself for each of the common toolchain commands, such as swift, swiftc, clang, etc. This mechanism will allows swiftly to proxy those command invocations to a selected toolchain at the time of invocation. A toolchain can be selected in these ways in order of precedence: * Special toolchain selectors among the regular tool command-line arguments (e.g. `swift build +5.10.1`) with the special '+' prefix * The presence of a .swift-version file in the current working directory, or ancestor directory, with the required toolchain version -* The swiftly default (in-use) toolchain set in the config.json by `swiftly install` or `swiftly use` commands +* The swiftly default (in-use) toolchain set in the swftly config.json by `swiftly install` or `swiftly use` commands + +If swiftly cannot find an installed toolchain that matches the selection then it fails with an error and instructions how to use `swiftly install` to satisfy the selection next time. + +#### Resolve selected toolchain + +For cases where the physical toolchain must be located, such as references specific header files, or shared libraries that are not proxied by swiftly there is a method to resolve the currently selected toolchain to its physical location using `swiftly use`. + +``` +swiftly use --location +``` -If swiftly cannot find an installed toolchain that matches the request then it fails with an error and instructions how to use `swiftly install` to fulfill the request. +This command will provide the full path to the directory where the selected toolchain is installed to standard output if such a toolchain exists. An external tool can directly navigate to the resources that it requires. For external tools that manage long-lived processes from the toolchain, such as the language server, and lldb, this command can be used in a poll to detect cases where the processes should be restarted. ## Detailed Design From ed68a9e15d9ae2a1ea1de643c59c851871dfbe2e Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Wed, 21 Aug 2024 08:17:00 -0400 Subject: [PATCH 05/26] add more details about the selector prefix, and methods to escape --- DESIGN.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/DESIGN.md b/DESIGN.md index 2ff4bf7b..2c419d5f 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -295,6 +295,8 @@ Swiftly will create a set of symbolic links in its SWIFTLY_BIN_DIR during instal If swiftly cannot find an installed toolchain that matches the selection then it fails with an error and instructions how to use `swiftly install` to satisfy the selection next time. +A few notes about the '+' prefix. First, if a literal '+' prefix should be sent directly to the tool as an argument then it is escaped by doubling it with '++'. An argument with only '++++' is ignored entirely, but any additional arguments are sent directly to the tool without any further inspection of their prefixes. This is analogous to the special '--' token that certain argument parsers accept so that they don't interpret anything following that token as command flags or options. + #### Resolve selected toolchain For cases where the physical toolchain must be located, such as references specific header files, or shared libraries that are not proxied by swiftly there is a method to resolve the currently selected toolchain to its physical location using `swiftly use`. From c745f4cd33f6fd9ea4252a35e47edd320d17ec86 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Thu, 29 Aug 2024 13:36:51 -0400 Subject: [PATCH 06/26] Restructure the PR to move the selector syntax from the proxies to a new swiftly run command --- DESIGN.md | 42 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 36 insertions(+), 6 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index 2c419d5f..e3aa5efb 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -24,7 +24,7 @@ This document contains the high level design of swiftly. Not all features have b Swiftly helps you to easily install different Swift toolchains locally on your account. It also provides a single path where you can run the tools in the currently selected toolchain. Toolchain selection is [configurable](#toolchain-selection) using different mechanisms. -Note that swiftly is *not* a virtual toolchain in itself since there are cases where it cannot behave as a self-contained Swift toolchain. For example, there can be external dependencies on specific files, such as headers or libraries, far too many and variable between toolchain versions to be managed by swiftly. Also, for long-lived processes, there is no way to gracefully restart them without help from the client. +Note that swiftly is *not* a virtual toolchain in itself since there are cases where it cannot behave as a self-contained Swift toolchain. For example, there can be external dependencies on specific files, such as headers or libraries. There are far too many files that change between toolchain versions to be managed by swiftly. Also, for long-lived processes, there is no way to gracefully restart them without help from the client. ## Installation of swiftly @@ -67,7 +67,7 @@ A simple setup for managing the toolchains could look like this: The toolchains (i.e. the contents of a given Swift download tarball) would be contained in the toolchains directory, each named according to the major/minor/patch version. `config.json` would contain any required metadata (e.g. the latest Swift version, which toolchain is selected, etc.). If pulling in Foundation to use `JSONEncoder`/`JSONDecoder` (or some other JSON tool) would be a problem, we could also use something simpler. -The `~/.local/bin` directory would include symlinks pointing to swiftly itself. When these binaries are run swiftly proxies them to the requested toolchain, or a default. +The `~/.local/bin` directory would include symlinks pointing to swiftly itself. When the proxies binaries are executed swiftly proxies them to the requested toolchain, or the default. This is all very similar to how rustup does things, but I figure there's no need to reinvent the wheel here. @@ -289,24 +289,54 @@ This command checks to see if there are new versions of `swiftly` itself and upg Swiftly will create a set of symbolic links in its SWIFTLY_BIN_DIR during installation that point to the swiftly binary itself for each of the common toolchain commands, such as swift, swiftc, clang, etc. This mechanism will allows swiftly to proxy those command invocations to a selected toolchain at the time of invocation. A toolchain can be selected in these ways in order of precedence: -* Special toolchain selectors among the regular tool command-line arguments (e.g. `swift build +5.10.1`) with the special '+' prefix * The presence of a .swift-version file in the current working directory, or ancestor directory, with the required toolchain version * The swiftly default (in-use) toolchain set in the swftly config.json by `swiftly install` or `swiftly use` commands If swiftly cannot find an installed toolchain that matches the selection then it fails with an error and instructions how to use `swiftly install` to satisfy the selection next time. -A few notes about the '+' prefix. First, if a literal '+' prefix should be sent directly to the tool as an argument then it is escaped by doubling it with '++'. An argument with only '++++' is ignored entirely, but any additional arguments are sent directly to the tool without any further inspection of their prefixes. This is analogous to the special '--' token that certain argument parsers accept so that they don't interpret anything following that token as command flags or options. - #### Resolve selected toolchain For cases where the physical toolchain must be located, such as references specific header files, or shared libraries that are not proxied by swiftly there is a method to resolve the currently selected toolchain to its physical location using `swiftly use`. ``` -swiftly use --location +swiftly use --print-location ``` This command will provide the full path to the directory where the selected toolchain is installed to standard output if such a toolchain exists. An external tool can directly navigate to the resources that it requires. For external tools that manage long-lived processes from the toolchain, such as the language server, and lldb, this command can be used in a poll to detect cases where the processes should be restarted. +#### Run with a selected toolchain + +There are cases where you might want to run an arbitrary command using a selected toolchain. An example could be that you want to build something with CMake. + +``` +# CMake +swiftly run cmake -G ninja +swiftly run ninja build + +# Autoconf +swiftly run ./configure +swiftly run make +``` + +Swiftly adjusts certain environment variables, such as prefixing the PATH to the selected toolchain directory, and setting the CC and CXX variables to the locations of clang and clang++ in those toolchains so that the build tools use them. If you want to explicitly specify a toolchain for the command you can do that with a selector notation like this: + +``` +swiftly run swift build +5.10.1 # Runs swift build with the 5.10.1 toolchain +``` + +A few notes about the '+' prefix. First, if a literal '+' prefix should be sent directly to the tool as an argument then it is escaped by doubling it with '++'. An argument with only '++++' is ignored entirely, but any additional arguments are sent directly to the command without any further inspection of their prefixes. This is analogous to the special '--' token that certain argument parsers accept so that they don't interpret anything following that token as command flags or options. + +If the selected toolchain is not installed then swiftly will exit with a message indicating that you need to run `swiftly install x.y.z` to install it. However, if you enter a special `+install` token then swiftly will automatically download and install the toolchain if it isn't already present. + +``` +# Download and install the latest main snapshot toolchain and run 'swift build' to build the package with it. +swiftly run swift build +main-snapshot +install + +# Generate makefiles with the latest released Swift toolchain, download and install it if necessary +swiftly run +latest +install cmake -G "Unix Makefile" +swiftly run +latest make +``` + ## Detailed Design Swiftly itself will be a SPM project consisting of several executable products, one per supported platform, and all of these will share the core module that handles argument parsing, printing help information, and dispatching commands. Each platform’s executable will be built to statically link the stdlib so that they can be run without having installed Swift first. From 560236d788eeaa9f0ea7207970caa43f7093e8c8 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Wed, 4 Sep 2024 15:40:46 -0400 Subject: [PATCH 07/26] Implement proxy mechanism with dynamic toolchain selection Change the nature of the swiftly symlinks so that they point to the swiftly executable at install time. These do not change when new toolchains are used. Toolchain selection happens each time when the proxies are run. The proxies are created for a well-known set of toolchain binaries that are constant for a wide variety of toolchain versions and platforms. Add support for .swift-version files for toolchain selection. Update the use command so that it can point out which toolchain is in use based on context, such as swift version files that are located in the current working directory or above. The fallback selection comes from the global default configuration's 'inUse' setting. When querying for what's in use the global default is shown with the "(default)" tag. If the in-use toolchain is selected by a swift-version file the path to that file is displayed. Provide a print location flag to the use subcommand that can print the file path of the toolchain that is in use in the current context. When using a new toolchain, depending on whether a swift version is selecting the current one, update the swift version file with the selected toolchain version. If no swift version file can be located, attempt to create a new one at the top of the git worktree. If there is no git worktree, then fallback to updating the global default in the configuration. Provide a global default flag for the use subcommand so that only the global default in-use toolchain is considered and not any of the swift version files. --- Sources/LinuxPlatform/Linux.swift | 86 +------------ Sources/MacOSPlatform/MacOS.swift | 96 +------------- Sources/Swiftly/Init.swift | 60 ++++++++- Sources/Swiftly/Install.swift | 7 +- Sources/Swiftly/Proxy.swift | 64 ++++++++++ Sources/Swiftly/Swiftly.swift | 1 - Sources/Swiftly/Uninstall.swift | 3 +- Sources/Swiftly/Use.swift | 166 +++++++++++++++++++++--- Sources/SwiftlyCore/Platform.swift | 29 +++-- Tests/SwiftlyTests/E2ETests.swift | 12 -- Tests/SwiftlyTests/PlatformTests.swift | 41 ------ Tests/SwiftlyTests/SwiftlyTests.swift | 20 ++- Tests/SwiftlyTests/UseTests.swift | 167 ++++++++++--------------- 13 files changed, 370 insertions(+), 382 deletions(-) create mode 100644 Sources/Swiftly/Proxy.swift diff --git a/Sources/LinuxPlatform/Linux.swift b/Sources/LinuxPlatform/Linux.swift index 7a96a5ec..d9a2e025 100644 --- a/Sources/LinuxPlatform/Linux.swift +++ b/Sources/LinuxPlatform/Linux.swift @@ -295,86 +295,6 @@ public struct Linux: Platform { try FileManager.default.removeItem(at: toolchainDir) } - public func use(_ toolchain: ToolchainVersion, currentToolchain: ToolchainVersion?) throws -> Bool { - let toolchainBinURL = self.swiftlyToolchainsDir - .appendingPathComponent(toolchain.name, isDirectory: true) - .appendingPathComponent("usr", isDirectory: true) - .appendingPathComponent("bin", isDirectory: true) - - if !FileManager.default.fileExists(atPath: toolchainBinURL.path) { - return false - } - - // Delete existing symlinks from previously in-use toolchain. - if let currentToolchain { - try self.unUse(currentToolchain: currentToolchain) - } - - // Ensure swiftly doesn't overwrite any existing executables without getting confirmation first. - let swiftlyBinDirContents = try FileManager.default.contentsOfDirectory(atPath: self.swiftlyBinDir.path) - let toolchainBinDirContents = try FileManager.default.contentsOfDirectory(atPath: toolchainBinURL.path) - let willBeOverwritten = Set(toolchainBinDirContents).intersection(swiftlyBinDirContents) - if !willBeOverwritten.isEmpty { - SwiftlyCore.print("The following existing executables will be overwritten:") - - for executable in willBeOverwritten { - SwiftlyCore.print(" \(self.swiftlyBinDir.appendingPathComponent(executable).path)") - } - - let proceed = SwiftlyCore.readLine(prompt: "Proceed? (y/n)") ?? "n" - - guard proceed == "y" else { - SwiftlyCore.print("Aborting use") - return false - } - } - - for executable in toolchainBinDirContents { - let linkURL = self.swiftlyBinDir.appendingPathComponent(executable) - let executableURL = toolchainBinURL.appendingPathComponent(executable) - - // Deletion confirmed with user above. - try linkURL.deleteIfExists() - - try FileManager.default.createSymbolicLink( - atPath: linkURL.path, - withDestinationPath: executableURL.path - ) - } - - return true - } - - public func unUse(currentToolchain: ToolchainVersion) throws { - let currentToolchainBinURL = self.swiftlyToolchainsDir - .appendingPathComponent(currentToolchain.name, isDirectory: true) - .appendingPathComponent("usr", isDirectory: true) - .appendingPathComponent("bin", isDirectory: true) - - for existingExecutable in try FileManager.default.contentsOfDirectory(atPath: currentToolchainBinURL.path) { - guard existingExecutable != "swiftly" else { - continue - } - - let url = self.swiftlyBinDir.appendingPathComponent(existingExecutable) - let vals = try url.resourceValues(forKeys: [.isSymbolicLinkKey]) - - guard let islink = vals.isSymbolicLink, islink else { - throw Error(message: "Found executable not managed by swiftly in SWIFTLY_BIN_DIR: \(url.path)") - } - let symlinkDest = url.resolvingSymlinksInPath() - guard symlinkDest.deletingLastPathComponent() == currentToolchainBinURL else { - throw Error(message: "Found symlink that points to non-swiftly managed executable: \(symlinkDest.path)") - } - - try self.swiftlyBinDir.appendingPathComponent(existingExecutable).deleteIfExists() - } - } - - public func listAvailableSnapshots(version _: String?) async -> [Snapshot] { - [] - } - public func getExecutableName() -> String { #if arch(x86_64) let arch = "x86_64" @@ -387,8 +307,6 @@ public struct Linux: Platform { return "swiftly-\(arch)-unknown-linux-gnu" } - public func currentToolchain() throws -> ToolchainVersion? { nil } - public func getTempFilePath() -> URL { FileManager.default.temporaryDirectory.appendingPathComponent("swiftly-\(UUID())") } @@ -605,5 +523,9 @@ public struct Linux: Platform { return "/bin/bash" } + public func findToolchainLocation(_ toolchain: ToolchainVersion) -> URL { + self.swiftlyToolchainsDir.appendingPathComponent("\(toolchain.name)") + } + public static let currentPlatform: any Platform = Linux() } diff --git a/Sources/MacOSPlatform/MacOS.swift b/Sources/MacOSPlatform/MacOS.swift index a7c0dd2f..218bbdec 100644 --- a/Sources/MacOSPlatform/MacOS.swift +++ b/Sources/MacOSPlatform/MacOS.swift @@ -101,102 +101,10 @@ public struct MacOS: Platform { try? runProgram("pkgutil", "--volume", homedir, "--forget", pkgInfo.CFBundleIdentifier) } - public func use(_ toolchain: ToolchainVersion, currentToolchain: ToolchainVersion?) throws -> Bool { - let toolchainBinURL = self.swiftlyToolchainsDir - .appendingPathComponent(toolchain.identifier + ".xctoolchain", isDirectory: true) - .appendingPathComponent("usr", isDirectory: true) - .appendingPathComponent("bin", isDirectory: true) - - if !FileManager.default.fileExists(atPath: toolchainBinURL.path) { - return false - } - - // Delete existing symlinks from previously in-use toolchain. - if let currentToolchain { - try self.unUse(currentToolchain: currentToolchain) - } - - // Ensure swiftly doesn't overwrite any existing executables without getting confirmation first. - let swiftlyBinDirContents = try FileManager.default.contentsOfDirectory(atPath: self.swiftlyBinDir.path) - let toolchainBinDirContents = try FileManager.default.contentsOfDirectory(atPath: toolchainBinURL.path) - let willBeOverwritten = Set(toolchainBinDirContents).intersection(swiftlyBinDirContents) - if !willBeOverwritten.isEmpty { - SwiftlyCore.print("The following existing executables will be overwritten:") - - for executable in willBeOverwritten { - SwiftlyCore.print(" \(self.swiftlyBinDir.appendingPathComponent(executable).path)") - } - - let proceed = SwiftlyCore.readLine(prompt: "Proceed? (y/n)") ?? "n" - - guard proceed == "y" else { - SwiftlyCore.print("Aborting use") - return false - } - } - - for executable in toolchainBinDirContents { - let linkURL = self.swiftlyBinDir.appendingPathComponent(executable) - let executableURL = toolchainBinURL.appendingPathComponent(executable) - - // Deletion confirmed with user above. - try linkURL.deleteIfExists() - - try FileManager.default.createSymbolicLink( - atPath: linkURL.path, - withDestinationPath: executableURL.path - ) - } - - SwiftlyCore.print(""" - NOTE: On macOS it is possible that the shell will pick up the system Swift on the path - instead of the one that swiftly has installed for you. You can run the 'hash -r' - command to update the shell with the latest PATHs. - - hash -r - - """ - ) - - return true - } - - public func unUse(currentToolchain: ToolchainVersion) throws { - let currentToolchainBinURL = self.swiftlyToolchainsDir - .appendingPathComponent(currentToolchain.identifier + ".xctoolchain", isDirectory: true) - .appendingPathComponent("usr", isDirectory: true) - .appendingPathComponent("bin", isDirectory: true) - - for existingExecutable in try FileManager.default.contentsOfDirectory(atPath: currentToolchainBinURL.path) { - guard existingExecutable != "swiftly" else { - continue - } - - let url = self.swiftlyBinDir.appendingPathComponent(existingExecutable) - let vals = try url.resourceValues(forKeys: [URLResourceKey.isSymbolicLinkKey]) - - guard let islink = vals.isSymbolicLink, islink else { - throw Error(message: "Found executable not managed by swiftly in SWIFTLY_BIN_DIR: \(url.path)") - } - let symlinkDest = url.resolvingSymlinksInPath() - guard symlinkDest.deletingLastPathComponent() == currentToolchainBinURL else { - throw Error(message: "Found symlink that points to non-swiftly managed executable: \(symlinkDest.path)") - } - - try self.swiftlyBinDir.appendingPathComponent(existingExecutable).deleteIfExists() - } - } - - public func listAvailableSnapshots(version _: String?) async -> [Snapshot] { - [] - } - public func getExecutableName() -> String { "swiftly-macos-osx" } - public func currentToolchain() throws -> ToolchainVersion? { nil } - public func getTempFilePath() -> URL { FileManager.default.temporaryDirectory.appendingPathComponent("swiftly-\(UUID()).pkg") } @@ -226,5 +134,9 @@ public struct MacOS: Platform { return "/bin/zsh" } + public func findToolchainLocation(_ toolchain: ToolchainVersion) -> URL { + self.swiftlyToolchainsDir.appendingPathComponent("\(toolchain.identifier).xctoolchain") + } + public static let currentPlatform: any Platform = MacOS() } diff --git a/Sources/Swiftly/Init.swift b/Sources/Swiftly/Init.swift index 5d86e95e..8b89438d 100644 --- a/Sources/Swiftly/Init.swift +++ b/Sources/Swiftly/Init.swift @@ -54,6 +54,24 @@ internal struct Init: SwiftlyCommand { } } + // Ensure swiftly doesn't overwrite any existing executables without getting confirmation first. + let swiftlyBinDir = Swiftly.currentPlatform.swiftlyBinDir + let swiftlyBinDirContents = try FileManager.default.contentsOfDirectory(atPath: swiftlyBinDir.path) + let willBeOverwritten = Set(proxyList + ["swiftly"]).intersection(swiftlyBinDirContents) + if !willBeOverwritten.isEmpty && !overwrite { + SwiftlyCore.print("The following existing executables will be overwritten:") + + for executable in willBeOverwritten { + SwiftlyCore.print(" \(swiftlyBinDir.appendingPathComponent(executable).path)") + } + + let proceed = SwiftlyCore.readLine(prompt: "Proceed? [y/N]") ?? "n" + + guard proceed == "y" else { + throw Error(message: "Swiftly installation has been cancelled") + } + } + let shell = if let s = ProcessInfo.processInfo.environment["SHELL"] { s } else { @@ -113,11 +131,7 @@ internal struct Init: SwiftlyCommand { SwiftlyCore.print("Moving swiftly into the installation directory...") if swiftlyBin.fileExists() { - if !overwrite { - throw Error(message: "Swiftly binary already exists. You can try again with `--overwrite` to replace it.") - } else { - try FileManager.default.removeItem(at: swiftlyBin) - } + try FileManager.default.removeItem(at: swiftlyBin) } do { @@ -128,6 +142,30 @@ internal struct Init: SwiftlyCommand { } } + // Don't create the proxies in the tests + if !cmd.path.hasSuffix("xctest") { + SwiftlyCore.print("Setting up toolchain proxies...") + + let proxyTo = if systemManaged { + cmd.path + } else { + swiftlyBin.path + } + + for p in proxyList { + let proxy = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent(p) + + if proxy.fileExists() { + try FileManager.default.removeItem(at: proxy) + } + + try FileManager.default.createSymbolicLink( + atPath: proxy.path, + withDestinationPath: proxyTo + ) + } + } + if overwrite || !FileManager.default.fileExists(atPath: envFile.path) { SwiftlyCore.print("Creating shell environment file for the user...") var env = "" @@ -201,7 +239,19 @@ internal struct Init: SwiftlyCommand { SwiftlyCore.print(""" To begin using installed swiftly from your current shell, first run the following command: \(sourceLine) + + """) + +#if os(macOS) + SwiftlyCore.print(""" + NOTE: On macOS it is possible that the shell will pick up the system Swift on the path + instead of the one that swiftly has installed for you. You can run the 'hash -r' + command to update the shell with the latest PATHs. + + hash -r + """) +#endif } } } diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index 63f82c1d..a618b5b3 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -77,7 +77,7 @@ struct Install: SwiftlyCommand { let selector = try ToolchainSelector(parsing: self.version) SwiftlyCore.httpClient.githubToken = self.token - let toolchainVersion = try await self.resolve(selector: selector) + let toolchainVersion = try await Self.resolve(selector: selector) var config = try Config.load() let postInstallScript = try await Self.execute( version: toolchainVersion, @@ -218,7 +218,8 @@ struct Install: SwiftlyCommand { // If this is the first installed toolchain, mark it as in-use regardless of whether the // --use argument was provided. if useInstalledToolchain || config.inUse == nil { - try await Use.execute(version, &config) + // TODO: consider adding the global default option to this commands flags + try await Use.execute(version, false, &config) } SwiftlyCore.print("\(version) installed successfully!") @@ -227,7 +228,7 @@ struct Install: SwiftlyCommand { /// Utilize the GitHub API along with the provided selector to select a toolchain for install. /// TODO: update this to use an official swift.org API - func resolve(selector: ToolchainSelector) async throws -> ToolchainVersion { + static func resolve(selector: ToolchainSelector) async throws -> ToolchainVersion { switch selector { case .latest: SwiftlyCore.print("Fetching the latest stable Swift release...") diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift new file mode 100644 index 00000000..8d9c9c60 --- /dev/null +++ b/Sources/Swiftly/Proxy.swift @@ -0,0 +1,64 @@ +import Foundation +import SwiftlyCore + +// This is the allowed list of executables that we will proxy +let proxyList = [ + "clang", + "lldb", + "lldb-dap", + "lldb-server", + "clang++", + "sourcekit-lsp", + "clangd", + "swift", + "docc", + "swiftc", + "lld", + "llvm-ar", + "plutil", + "repl_swift", + "wasm-ld", +] + +@main +public enum Proxy { + static func main() async throws { + do { + let zero = CommandLine.arguments[0] + guard let binName = zero.components(separatedBy: "/").last else { + fatalError("Could not determine the binary name for proxying") + } + + guard proxyList.contains(binName) else { + // Treat this as a swiftly invocation + await Swiftly.main() + return + } + + let config = try Config.load() + let toolchain: ToolchainVersion + + if let (selectedToolchain, versionFile, selector) = try swiftToolchainSelection(config: config) { + guard let selectedToolchain = selectedToolchain else { + if let versionFile = versionFile { + throw if let selector = selector { + Error(message: "No installed swift toolchain matches the version \(selector) from \(versionFile). You can try installing one with `swiftly install \(selector)`.") + } else { + Error(message: "Swift version file is malformed and cannot be used to select a swift toolchain: \(versionFile)") + } + } + fatalError("error in toolchain selection logic") + } + + toolchain = selectedToolchain + } else { + throw Error(message: "No swift toolchain could be determined either from a .swift-version file, or the default. You can try using `swiftly use ` to set it.") + } + + try await Swiftly.currentPlatform.proxy(toolchain, binName, Array(CommandLine.arguments[1...])) + } catch { + SwiftlyCore.print("\(error)") + exit(1) + } + } +} diff --git a/Sources/Swiftly/Swiftly.swift b/Sources/Swiftly/Swiftly.swift index ce141b37..caa473ed 100644 --- a/Sources/Swiftly/Swiftly.swift +++ b/Sources/Swiftly/Swiftly.swift @@ -14,7 +14,6 @@ public struct GlobalOptions: ParsableArguments { public init() {} } -@main public struct Swiftly: SwiftlyCommand { public static var configuration = CommandConfiguration( abstract: "A utility for installing and managing Swift toolchains.", diff --git a/Sources/Swiftly/Uninstall.swift b/Sources/Swiftly/Uninstall.swift index a763cd4f..1bfb8ff2 100644 --- a/Sources/Swiftly/Uninstall.swift +++ b/Sources/Swiftly/Uninstall.swift @@ -100,10 +100,9 @@ struct Uninstall: SwiftlyCommand { ?? config.listInstalledToolchains(selector: .latest).filter({ !toolchains.contains($0) }).max() ?? config.installedToolchains.filter({ !toolchains.contains($0) }).max() { - try await Use.execute(toUse, &config) + try await Use.execute(toUse, true, &config) } else { // If there are no more toolchains installed, just unuse the currently active toolchain. - try Swiftly.currentPlatform.unUse(currentToolchain: toolchain) config.inUse = nil try config.save() } diff --git a/Sources/Swiftly/Use.swift b/Sources/Swiftly/Use.swift index b1cc9238..740ca0ab 100644 --- a/Sources/Swiftly/Use.swift +++ b/Sources/Swiftly/Use.swift @@ -1,4 +1,5 @@ import ArgumentParser +import Foundation import SwiftlyCore internal struct Use: SwiftlyCommand { @@ -6,6 +7,12 @@ internal struct Use: SwiftlyCommand { abstract: "Set the active toolchain. If no toolchain is provided, print the currently in-use toolchain, if any." ) + @Flag(name: .shortAndLong, help: "Print the location of the in-use toolchain. This is valid only when there is no toolchain argument.") + var printLocation: Bool = false + + @Flag(name: .shortAndLong, help: "Use the global default, ignoring any .swift-version files.") + var globalDefault: Bool = false + @Argument(help: ArgumentHelp( "The toolchain to use.", discussion: """ @@ -46,13 +53,44 @@ internal struct Use: SwiftlyCommand { try validateSwiftly() var config = try Config.load() + // This is the bare use command where we print the selected toolchain version (or the path to it) guard let toolchain = self.toolchain else { - if let inUse = config.inUse { - SwiftlyCore.print("\(inUse) (in use)") + let selected = try swiftToolchainSelection(config: config, globalDefault: self.globalDefault) + + guard let selected = selected else { + // No toolchain selected, so we just output nothing + return + } + + let (selectedVersion, versionFile, selector) = selected + + if let versionFile = versionFile, selector == nil { + throw Error(message: "Swift version file is malformed and cannot be used to select a swift toolchain: \(versionFile)") + } + + guard let selectedVersion = selectedVersion else { + fatalError("error in toolchain selection logic") + } + + if self.printLocation { + // Print the toolchain location and exit + SwiftlyCore.print("\(Swiftly.currentPlatform.findToolchainLocation(selectedVersion).path)") + return + } + + if let versionFile = versionFile { + SwiftlyCore.print("\(selectedVersion) (\(versionFile.path))") + } else { + SwiftlyCore.print("\(selectedVersion) (default)") } + return } + guard !self.printLocation else { + throw Error(message: "The print location flag cannot be used with a toolchain version.") + } + let selector = try ToolchainSelector(parsing: toolchain) guard let toolchain = config.listInstalledToolchains(selector: selector).max() else { @@ -60,29 +98,125 @@ internal struct Use: SwiftlyCommand { return } - try await Self.execute(toolchain, &config) + try await Self.execute(toolchain, self.globalDefault, &config) } - /// Use a toolchain. This method modifies and saves the input config. - internal static func execute(_ toolchain: ToolchainVersion, _ config: inout Config) async throws { - let previousToolchain = config.inUse + /// Use a toolchain. This method can modify and save the input config. + internal static func execute(_ toolchain: ToolchainVersion, _ globalDefault: Bool, _ config: inout Config) async throws { + let previousToolchain = try swiftToolchainSelection(config: config, globalDefault: globalDefault) - guard toolchain != previousToolchain else { - SwiftlyCore.print("\(toolchain) is already in use") - return + if let (selectedVersion, _, _) = previousToolchain { + guard selectedVersion != toolchain else { + SwiftlyCore.print("\(toolchain) is already in use") + return + } } - guard try Swiftly.currentPlatform.use(toolchain, currentToolchain: previousToolchain) else { - return + if let (_, versionFile, _) = previousToolchain, let versionFile = versionFile { + try toolchain.name.write(to: versionFile, atomically: true, encoding: .utf8) + } else if let newVersionFile = findNewVersionFile(), !globalDefault { + try toolchain.name.write(to: newVersionFile, atomically: true, encoding: .utf8) + } else { + config.inUse = toolchain + try config.save() } - config.inUse = toolchain - try config.save() - var message = "Set the active toolchain to \(toolchain)" - if let previousToolchain { - message += " (was \(previousToolchain))" + var message = "Set the used toolchain to \(toolchain)" + if let (selectedVersion, _, _) = previousToolchain, + let selectedVersion = selectedVersion + { + message += " (was \(selectedVersion.name))" } SwiftlyCore.print(message) } + + internal static func findNewVersionFile() -> URL? { + var cwd = URL(fileURLWithPath: FileManager.default.currentDirectoryPath) + + while cwd.path != "" && cwd.path != "/" { + guard FileManager.default.fileExists(atPath: cwd.path) else { + break + } + + let gitDir = cwd.appendingPathComponent(".git") + + if FileManager.default.fileExists(atPath: gitDir.path) { + return cwd.appendingPathComponent(".swift-version") + } + + cwd = cwd.deletingLastPathComponent() + } + + return nil + } +} + +/// Returns the currently selected swift toolchain with optional details. +/// +/// Selection of a toolchain can be accomplished in a number of ways. There is the +/// the configuration's global default 'inUse' setting. This is the fallback selector +/// if there are no other selections. In this case the returned tuple will contain +/// only the selected toolchain version. None of the other values are provided. +/// +/// A toolchain can also be selected from a `.swift-version` file in the current +/// working directory, or an ancestor directory. The nearest version file is +/// returned as a URL if it is present, even if the file is malformed or it +/// doesn't select any of the installed toolchains. A well-formed version file +/// will additionally return the toolchain selector that it represents. Finally, +/// if that selector selects one of the installed toolchains then all three +/// values are returned. +/// +/// Note: if no swift version files are found at all then the return will be nil +/// +public func swiftToolchainSelection(config: Config, globalDefault: Bool = false) throws -> (ToolchainVersion?, URL?, ToolchainSelector?)? { + if !globalDefault { + var cwd = URL(fileURLWithPath: FileManager.default.currentDirectoryPath) + + while cwd.path != "" && cwd.path != "/" { + guard FileManager.default.fileExists(atPath: cwd.path) else { + break + } + + let svFile = cwd.appendingPathComponent(".swift-version") + + if FileManager.default.fileExists(atPath: svFile.path) { + let contents = try? String(contentsOf: svFile, encoding: .utf8) + + guard let contents = contents else { + return (nil, svFile, nil) + } + + guard !contents.isEmpty else { + return (nil, svFile, nil) + } + + let selectorString = contents.replacingOccurrences(of: "\n", with: "").replacingOccurrences(of: "\r", with: "") + let selector: ToolchainSelector? + do { + selector = try ToolchainSelector(parsing: selectorString) + } catch { + return (nil, svFile, nil) + } + + guard let selector = selector else { + return (nil, svFile, nil) + } + + guard let selectedToolchain = config.listInstalledToolchains(selector: selector).max() else { + return (nil, svFile, selector) + } + + return (selectedToolchain, svFile, selector) + } + + cwd = cwd.deletingLastPathComponent() + } + } + + if let inUse = config.inUse { + return (inUse, nil, nil) + } + + return nil } diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index 2eb0f833..6c8b1fb0 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -54,18 +54,6 @@ public protocol Platform { /// If this version is in use, the next latest version will be used afterwards. func uninstall(_ version: ToolchainVersion) throws - /// Select the toolchain associated with the given version. - /// Returns whether the selection was successful. - func use(_ version: ToolchainVersion, currentToolchain: ToolchainVersion?) throws -> Bool - - /// Clear the current active toolchain. - func unUse(currentToolchain: ToolchainVersion) throws - - /// Get a list of snapshot builds for the platform. If a version is specified, only - /// return snapshots associated with the version. - /// This will likely have a default implementation. - func listAvailableSnapshots(version: String?) async -> [Snapshot] - /// Get the name of the swiftly release binary. func getExecutableName() -> String @@ -98,6 +86,9 @@ public protocol Platform { /// Get the user's current login shell func getShell() async throws -> String + + /// Find the location where the toolchain should be installed. + func findToolchainLocation(_ toolchain: ToolchainVersion) -> URL } extension Platform { @@ -126,7 +117,16 @@ extension Platform { } #if os(macOS) || os(Linux) + public func proxy(_ toolchain: ToolchainVersion, _ command: String, _ arguments: [String]) async throws { + let cmd = self.findToolchainLocation(toolchain).appendingPathComponent("usr/bin/\(command)") + try self.runProgram([cmd.path] + arguments) + } + public func runProgram(_ args: String..., quiet: Bool = false) throws { + try self.runProgram([String](args), quiet: quiet) + } + + public func runProgram(_ args: [String], quiet: Bool = false) throws { let process = Process() process.executableURL = URL(fileURLWithPath: "/usr/bin/env") process.arguments = args @@ -150,6 +150,10 @@ extension Platform { } public func runProgramOutput(_ program: String, _ args: String...) async throws -> String? { + try await self.runProgramOutput(program, [String](args)) + } + + public func runProgramOutput(_ program: String, _ args: [String]) async throws -> String? { let process = Process() process.executableURL = URL(fileURLWithPath: "/usr/bin/env") process.arguments = [program] + args @@ -202,6 +206,7 @@ extension Platform { return true } + #endif } diff --git a/Tests/SwiftlyTests/E2ETests.swift b/Tests/SwiftlyTests/E2ETests.swift index 88cbe0b3..29ac97a0 100644 --- a/Tests/SwiftlyTests/E2ETests.swift +++ b/Tests/SwiftlyTests/E2ETests.swift @@ -61,18 +61,6 @@ final class E2ETests: SwiftlyTests { XCTAssertTrue(release >= ToolchainVersion.StableRelease(major: 5, minor: 8, patch: 0)) try await validateInstalledToolchains([installedToolchain], description: "install latest") - - if let envScript = envScript { - let whichCmd = if shell.hasSuffix("bash") { - "type -P swift" - } else { - "which swift" - } - - // Check that within a new shell, the swift version succeeds and is the version we expect - let whichSwift = (try? await Swiftly.currentPlatform.runProgramOutput(shell, "-c", ". \(envScript.path) ; \(whichCmd)")) ?? "" - XCTAssertTrue(whichSwift.hasPrefix(Swiftly.currentPlatform.swiftlyBinDir.path)) - } } } } diff --git a/Tests/SwiftlyTests/PlatformTests.swift b/Tests/SwiftlyTests/PlatformTests.swift index d9f07e61..01e5841c 100644 --- a/Tests/SwiftlyTests/PlatformTests.swift +++ b/Tests/SwiftlyTests/PlatformTests.swift @@ -73,45 +73,4 @@ final class PlatformTests: SwiftlyTests { XCTAssertEqual(0, toolchains.count) } } - - func testUse() async throws { - try await self.rollbackLocalChanges { - // GIVEN: toolchains have been downloaded, and installed - var (mockedToolchainFile, version) = try await self.mockToolchainDownload(version: "5.8.0") - try Swiftly.currentPlatform.install(from: mockedToolchainFile, version: version) - (mockedToolchainFile, version) = try await self.mockToolchainDownload(version: "5.6.3") - try Swiftly.currentPlatform.install(from: mockedToolchainFile, version: version) - // WHEN: one of the toolchains is used - var result = try Swiftly.currentPlatform.use(ToolchainVersion(parsing: "5.8.0"), currentToolchain: nil) - // THEN: there are symbolic links for the toolchain binaries in the bin dir that point to the toolchain - XCTAssertTrue(result) - var swiftLinkTarget = try? FileManager.default.destinationOfSymbolicLink(atPath: Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swift").path) - guard let target = swiftLinkTarget else { - throw Error(message: "swift symlink was not found") - } - XCTAssertTrue(target.contains("5.8")) - - // GIVEN: toolchains have been downloaded, installed, and a toolchain is in use - // WHEN: another toolchain is used - result = try Swiftly.currentPlatform.use(ToolchainVersion(parsing: "5.6.3"), currentToolchain: ToolchainVersion(parsing: "5.8.0")) - // THEN: there are symbolic links for the toolchain binaries in the bin dir that point to the toolchain - XCTAssertTrue(result) - swiftLinkTarget = try? FileManager.default.destinationOfSymbolicLink(atPath: Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swift").path) - guard let target2 = swiftLinkTarget else { - throw Error(message: "swift symlink was not found") - } - XCTAssertTrue(target2.contains("5.6.3")) - - // GIVEN: toolchains have been downloaded, installed, and a toolchain is in use - // WHEN: a toolchain is used that has not been installed - result = try Swiftly.currentPlatform.use(ToolchainVersion(parsing: "5.2.1"), currentToolchain: ToolchainVersion(parsing: "5.6.3")) - // THEN: the symbolic links remain the same - XCTAssertFalse(result) - swiftLinkTarget = try? FileManager.default.destinationOfSymbolicLink(atPath: Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swift").path) - guard let target3 = swiftLinkTarget else { - throw Error(message: "swift symlink was not found") - } - XCTAssertTrue(target3.contains("5.6.3")) - } - } } diff --git a/Tests/SwiftlyTests/SwiftlyTests.swift b/Tests/SwiftlyTests/SwiftlyTests.swift index 3fed966a..0b4e9656 100644 --- a/Tests/SwiftlyTests/SwiftlyTests.swift +++ b/Tests/SwiftlyTests/SwiftlyTests.swift @@ -140,7 +140,7 @@ class SwiftlyTests: XCTestCase { } class func getTestHomePath(name: String) -> URL { - URL(fileURLWithPath: FileManager.default.currentDirectoryPath).appendingPathComponent(name, isDirectory: true) + FileManager.default.temporaryDirectory.appendingPathComponent("swiftly-tests-\(name)-\(UUID())") } /// Create a fresh swiftly home directory, populate it with a base config, and run the provided closure. @@ -168,6 +168,13 @@ class SwiftlyTests: XCTestCase { let config = try await self.baseTestConfig() try config.save() + let cwd = FileManager.default.currentDirectoryPath + defer { + FileManager.default.changeCurrentDirectoryPath(cwd) + } + + FileManager.default.changeCurrentDirectoryPath(testHome.path) + try await f() } @@ -292,17 +299,6 @@ class SwiftlyTests: XCTestCase { func validateInUse(expected: ToolchainVersion?) async throws { let config = try Config.load() XCTAssertEqual(config.inUse, expected) - - let executable = SwiftExecutable(path: Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swift")) - - XCTAssertEqual(executable.exists(), expected != nil) - - guard let expected else { - return - } - - let inUseVersion = try await executable.version() - XCTAssertEqual(inUseVersion, expected) } /// Validate that all of the provided toolchains have been installed. diff --git a/Tests/SwiftlyTests/UseTests.swift b/Tests/SwiftlyTests/UseTests.swift index 724e2cce..604af79a 100644 --- a/Tests/SwiftlyTests/UseTests.swift +++ b/Tests/SwiftlyTests/UseTests.swift @@ -9,15 +9,10 @@ final class UseTests: SwiftlyTests { /// Execute a `use` command with the provided argument. Then validate that the configuration is updated properly and /// the in-use swift executable prints the the provided expectedVersion. func useAndValidate(argument: String, expectedVersion: ToolchainVersion) async throws { - var use = try self.parseCommand(Use.self, ["use", argument]) + var use = try self.parseCommand(Use.self, ["use", "-g", argument]) try await use.run() XCTAssertEqual(try Config.load().inUse, expectedVersion) - - let toolchainVersion = try self.getMockedToolchainVersion( - at: Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swift") - ) - XCTAssertEqual(toolchainVersion, expectedVersion) } /// Tests that the `use` command can switch between installed stable release toolchains. @@ -171,13 +166,13 @@ final class UseTests: SwiftlyTests { /// Tests that the `use` command gracefully exits when executed before any toolchains have been installed. func testUseNoInstalledToolchains() async throws { try await self.withMockedHome(homeName: Self.homeName, toolchains: []) { - var use = try self.parseCommand(Use.self, ["use", "latest"]) + var use = try self.parseCommand(Use.self, ["use", "-g", "latest"]) try await use.run() var config = try Config.load() XCTAssertEqual(config.inUse, nil) - use = try self.parseCommand(Use.self, ["use", "5.6.0"]) + use = try self.parseCommand(Use.self, ["use", "-g", "5.6.0"]) try await use.run() config = try Config.load() @@ -213,117 +208,81 @@ final class UseTests: SwiftlyTests { } } - /// Tests that the `use` command symlinks all of the executables provided in a toolchain and removes any existing - /// symlinks from the previously active toolchain. - func testOldSymlinksRemoved() async throws { - try await self.withMockedHome(homeName: Self.homeName, toolchains: Self.allToolchains) { - let spec = [ - ToolchainVersion(major: 1, minor: 2, patch: 3): ["a", "b"], - ToolchainVersion(major: 2, minor: 3, patch: 4): ["b", "c", "d"], - ToolchainVersion(major: 3, minor: 4, patch: 5): ["a", "c", "d", "e"], - ] - - for (toolchain, files) in spec { - try await self.installMockedToolchain(toolchain: toolchain, executables: files) - } - - // Add an unrelated executable to the binary directory. - let existingFileName = "existing" - let existingExecutableURL = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent(existingFileName) - let data = Data("hello world\n".utf8) - try data.write(to: existingExecutableURL) - - for (toolchain, files) in spec { - var use = try self.parseCommand(Use.self, ["use", toolchain.name]) + /// Tests that running a use command without an argument prints the currently in-use toolchain. + func testPrintInUse() async throws { + let toolchains = [ + Self.newStable, + Self.newMainSnapshot, + Self.newReleaseSnapshot, + ] + try await self.withMockedHome(homeName: Self.homeName, toolchains: Set(toolchains)) { + for toolchain in toolchains { + var use = try self.parseCommand(Use.self, ["use", "-g", toolchain.name]) try await use.run() - // Verify that only the symlinks for the active toolchain remain. - let symlinks = try FileManager.default.contentsOfDirectory( - atPath: Swiftly.currentPlatform.swiftlyBinDir.path - ) - XCTAssertEqual(symlinks.sorted(), (files + [existingFileName]).sorted()) - - // Verify that any all the symlinks point to the right toolchain. - for file in files { - guard file != existingFileName else { - continue - } - let observedVersion = try self.getMockedToolchainVersion( - at: Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent(file) - ) - XCTAssertEqual(observedVersion, toolchain) - } - } - } - } - - /// Tests that any executables that already exist in SWIFTLY_BIN_DIR. - func testExistingExecutablesNotOverwritten() async throws { - try await self.withMockedHome(homeName: Self.homeName, toolchains: []) { - let existingExecutables = ["a", "b", "c"] - let existingText = "existing" - for fileName in existingExecutables { - let existingExecutableURL = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent(fileName) - let data = Data(existingText.utf8) - try data.write(to: existingExecutableURL) - } - - let toolchain = ToolchainVersion(major: 7, minor: 2, patch: 3) - try await self.installMockedToolchain( - toolchain: toolchain, - executables: ["a", "b", "c", "d", "e"] - ) - - var use = try self.parseCommand(Use.self, ["use", toolchain.name]) - let nOutput = try await use.runWithMockedIO(input: ["n"]) + var useEmpty = try self.parseCommand(Use.self, ["use", "-g"]) + var output = try await useEmpty.runWithMockedIO() - for exec in existingExecutables { - // Ensure we were prompted for each existing executable. - XCTAssert(nOutput.contains(where: { $0.contains(exec) })) - - // Ensure files were not overwritten. - let existingExecutableURL = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent(exec) - let contents = try String(contentsOf: existingExecutableURL, encoding: .utf8) - XCTAssertEqual(contents, existingText) - } - - let nConfig = try Config.load() - XCTAssertEqual(nConfig.inUse, nil) - - let yOutput = try await use.runWithMockedIO(input: ["y"]) + XCTAssert(output.contains(where: { $0.contains(String(describing: toolchain)) })) - // Ensure we were prompted for each existing executable. - for exec in existingExecutables { - XCTAssert(yOutput.contains(where: { $0.contains(exec) })) + useEmpty = try self.parseCommand(Use.self, ["use", "-g", "--print-location"]) + output = try await useEmpty.runWithMockedIO() - // Ensure files were overwritten. - let existingExecutableURL = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent(exec) - let contents = try String(contentsOf: existingExecutableURL, encoding: .utf8) - XCTAssertNotEqual(contents, existingText) + XCTAssert(output.contains(where: { $0.contains(Swiftly.currentPlatform.findToolchainLocation(toolchain).path) })) } - - let yConfig = try Config.load() - XCTAssertEqual(yConfig.inUse, toolchain) } } - /// Tests that running a use command without an argument prints the currently in-use toolchain. - func testPrintInUse() async throws { + /// Tests in-use toolchain selected by the .swift-version file. + func testSwiftVersionFile() async throws { let toolchains = [ Self.newStable, Self.newMainSnapshot, Self.newReleaseSnapshot, ] try await self.withMockedHome(homeName: Self.homeName, toolchains: Set(toolchains)) { - for toolchain in toolchains { - var use = try self.parseCommand(Use.self, ["use", toolchain.name]) - try await use.run() - - var useEmpty = try self.parseCommand(Use.self, ["use"]) - let output = try await useEmpty.runWithMockedIO() - - XCTAssert(output.contains(where: { $0.contains(String(describing: toolchain)) })) - } + let versionFile = URL(fileURLWithPath: FileManager.default.currentDirectoryPath).appendingPathComponent(".swift-version") + + // GIVEN: a directory with a swift version file that selects a particular toolchain + try Self.newStable.name.write(to: versionFile, atomically: true, encoding: .utf8) + // WHEN: checking which toolchain is selected with the use command + var useCmd = try self.parseCommand(Use.self, ["use"]) + var output = try await useCmd.runWithMockedIO() + // THEN: the output shows this toolchain is in use with this working directory + XCTAssert(output.contains(where: { $0.contains(Self.newStable.name) })) + + // GIVEN: a directory with a swift version file that selects a particular toolchain + // WHEN: using another toolchain version + useCmd = try self.parseCommand(Use.self, ["use", Self.newMainSnapshot.name]) + output = try await useCmd.runWithMockedIO() + // THEN: the swift version file is updated to this toolchain version + var versionFileContents = try String(contentsOf: versionFile, encoding: .utf8) + XCTAssertEqual(Self.newMainSnapshot.name, versionFileContents) + // THEN: the use command reports this toolchain to be in use + XCTAssert(output.contains(where: { $0.contains(Self.newMainSnapshot.name) })) + + // GIVEN: a directory with no swift version file at the top of a git repository + try FileManager.default.removeItem(atPath: versionFile.path) + let gitDir = URL(fileURLWithPath: FileManager.default.currentDirectoryPath).appendingPathComponent(".git") + try FileManager.default.createDirectory(atPath: gitDir.path, withIntermediateDirectories: false) + // WHEN: using a toolchain version + useCmd = try self.parseCommand(Use.self, ["use", Self.newReleaseSnapshot.name]) + try await useCmd.run() + // THEN: a swift version file is created + XCTAssert(FileManager.default.fileExists(atPath: versionFile.path)) + // THEN: the version file contains the specified version + versionFileContents = try String(contentsOf: versionFile, encoding: .utf8) + XCTAssertEqual(Self.newReleaseSnapshot.name, versionFileContents) + + // GIVEN: a directory with a swift version file at the top of a git repository + try "1.2.3".write(to: versionFile, atomically: true, encoding: .utf8) + // WHEN: using with a toolchain selector that can select more than one version, but matches one of the installed toolchains + let broadSelector = ToolchainSelector.stable(major: Self.newStable.asStableRelease!.major, minor: nil, patch: nil) + useCmd = try self.parseCommand(Use.self, ["use", broadSelector.description]) + try await useCmd.run() + // THEN: the swift version file is set to the specific toolchain version that was installed including major, minor, and patch + versionFileContents = try String(contentsOf: versionFile, encoding: .utf8) + XCTAssertEqual(Self.newStable.name, versionFileContents) } } } From ce0d27a7892df4d251ed0dcb9e726421817baac9 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Thu, 5 Sep 2024 15:09:42 -0400 Subject: [PATCH 08/26] Rewrite the select toolchain function with a type for the selection result and centralized error messages --- Sources/Swiftly/Proxy.swift | 23 ++++------ Sources/Swiftly/Use.swift | 90 +++++++++++++++++++------------------ 2 files changed, 54 insertions(+), 59 deletions(-) diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift index 8d9c9c60..45b3dcea 100644 --- a/Sources/Swiftly/Proxy.swift +++ b/Sources/Swiftly/Proxy.swift @@ -36,23 +36,16 @@ public enum Proxy { } let config = try Config.load() - let toolchain: ToolchainVersion - if let (selectedToolchain, versionFile, selector) = try swiftToolchainSelection(config: config) { - guard let selectedToolchain = selectedToolchain else { - if let versionFile = versionFile { - throw if let selector = selector { - Error(message: "No installed swift toolchain matches the version \(selector) from \(versionFile). You can try installing one with `swiftly install \(selector)`.") - } else { - Error(message: "Swift version file is malformed and cannot be used to select a swift toolchain: \(versionFile)") - } - } - fatalError("error in toolchain selection logic") - } + let (toolchain, result) = selectToolchain(config: config) - toolchain = selectedToolchain - } else { - throw Error(message: "No swift toolchain could be determined either from a .swift-version file, or the default. You can try using `swiftly use ` to set it.") + // Abort on any errors relating to swift version files + if case let .swiftVersionFile(_, error) = result, let error = error { + throw error + } + + guard let toolchain = toolchain else { + throw Error(message: "No swift toolchain could be selected from either from a .swift-version file, or the default. You can try using `swiftly install ` to install one.") } try await Swiftly.currentPlatform.proxy(toolchain, binName, Array(CommandLine.arguments[1...])) diff --git a/Sources/Swiftly/Use.swift b/Sources/Swiftly/Use.swift index 740ca0ab..ced9e4c9 100644 --- a/Sources/Swiftly/Use.swift +++ b/Sources/Swiftly/Use.swift @@ -55,21 +55,16 @@ internal struct Use: SwiftlyCommand { // This is the bare use command where we print the selected toolchain version (or the path to it) guard let toolchain = self.toolchain else { - let selected = try swiftToolchainSelection(config: config, globalDefault: self.globalDefault) + let (selectedVersion, result) = selectToolchain(config: config, globalDefault: self.globalDefault) - guard let selected = selected else { - // No toolchain selected, so we just output nothing - return - } - - let (selectedVersion, versionFile, selector) = selected - - if let versionFile = versionFile, selector == nil { - throw Error(message: "Swift version file is malformed and cannot be used to select a swift toolchain: \(versionFile)") + // Abort on any errors with the swift version files + if case let .swiftVersionFile(_, error) = result, let error = error { + throw error } guard let selectedVersion = selectedVersion else { - fatalError("error in toolchain selection logic") + // Return with nothing if there's no toolchain that is selected + return } if self.printLocation { @@ -78,12 +73,17 @@ internal struct Use: SwiftlyCommand { return } - if let versionFile = versionFile { - SwiftlyCore.print("\(selectedVersion) (\(versionFile.path))") - } else { - SwiftlyCore.print("\(selectedVersion) (default)") + var message = "\(selectedVersion)" + + switch result { + case let .swiftVersionFile(versionFile, _): + message += " (\(versionFile.path))" + case .globalDefault: + message += " (default)" } + SwiftlyCore.print(message) + return } @@ -103,16 +103,17 @@ internal struct Use: SwiftlyCommand { /// Use a toolchain. This method can modify and save the input config. internal static func execute(_ toolchain: ToolchainVersion, _ globalDefault: Bool, _ config: inout Config) async throws { - let previousToolchain = try swiftToolchainSelection(config: config, globalDefault: globalDefault) + let (selectedVersion, result) = selectToolchain(config: config, globalDefault: globalDefault) - if let (selectedVersion, _, _) = previousToolchain { + if let selectedVersion = selectedVersion { guard selectedVersion != toolchain else { SwiftlyCore.print("\(toolchain) is already in use") return } } - if let (_, versionFile, _) = previousToolchain, let versionFile = versionFile { + if case let .swiftVersionFile(versionFile, _) = result { + // We don't care in this case if there were any problems with the swift version files, just overwrite it with the new value try toolchain.name.write(to: versionFile, atomically: true, encoding: .utf8) } else if let newVersionFile = findNewVersionFile(), !globalDefault { try toolchain.name.write(to: newVersionFile, atomically: true, encoding: .utf8) @@ -122,9 +123,7 @@ internal struct Use: SwiftlyCommand { } var message = "Set the used toolchain to \(toolchain)" - if let (selectedVersion, _, _) = previousToolchain, - let selectedVersion = selectedVersion - { + if let selectedVersion = selectedVersion { message += " (was \(selectedVersion.name))" } @@ -152,24 +151,31 @@ internal struct Use: SwiftlyCommand { } } -/// Returns the currently selected swift toolchain with optional details. +public enum ToolchainSelectionResult { + case globalDefault + case swiftVersionFile(URL, Error?) +} + +/// Returns the currently selected swift toolchain, if any, with details of the selection. +/// +/// The first portion of the returned tuple is the version that was selected, which +/// can be nil if none can be selected. /// /// Selection of a toolchain can be accomplished in a number of ways. There is the /// the configuration's global default 'inUse' setting. This is the fallback selector -/// if there are no other selections. In this case the returned tuple will contain -/// only the selected toolchain version. None of the other values are provided. +/// if there are no other selections. The returned tuple will contain the default toolchain +/// version and the result will be .default. /// /// A toolchain can also be selected from a `.swift-version` file in the current -/// working directory, or an ancestor directory. The nearest version file is -/// returned as a URL if it is present, even if the file is malformed or it -/// doesn't select any of the installed toolchains. A well-formed version file -/// will additionally return the toolchain selector that it represents. Finally, -/// if that selector selects one of the installed toolchains then all three -/// values are returned. +/// working directory, or an ancestor directory. If it successfully selects a toolchain +/// then the result will be .swiftVersionFile with the URL of the file that made the +/// selection and the first item of the tuple is the selected toolchain version. /// -/// Note: if no swift version files are found at all then the return will be nil -/// -public func swiftToolchainSelection(config: Config, globalDefault: Bool = false) throws -> (ToolchainVersion?, URL?, ToolchainSelector?)? { +/// However, there are cases where the swift version file fails to select a toolchain. +/// If such a case happens then the toolchain version in the tuple will be nil, but the +/// result will be .swiftVersionFile and a detailed error about the problem. This error +/// can be thrown by the client, or ignored. +public func selectToolchain(config: Config, globalDefault: Bool = false) -> (ToolchainVersion?, ToolchainSelectionResult) { if !globalDefault { var cwd = URL(fileURLWithPath: FileManager.default.currentDirectoryPath) @@ -184,11 +190,11 @@ public func swiftToolchainSelection(config: Config, globalDefault: Bool = false) let contents = try? String(contentsOf: svFile, encoding: .utf8) guard let contents = contents else { - return (nil, svFile, nil) + return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file could not be read: \(svFile)"))) } guard !contents.isEmpty else { - return (nil, svFile, nil) + return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file is empty: \(svFile)"))) } let selectorString = contents.replacingOccurrences(of: "\n", with: "").replacingOccurrences(of: "\r", with: "") @@ -196,27 +202,23 @@ public func swiftToolchainSelection(config: Config, globalDefault: Bool = false) do { selector = try ToolchainSelector(parsing: selectorString) } catch { - return (nil, svFile, nil) + return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file is malformed: \(svFile) \(error)"))) } guard let selector = selector else { - return (nil, svFile, nil) + return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file is malformed: \(svFile)"))) } guard let selectedToolchain = config.listInstalledToolchains(selector: selector).max() else { - return (nil, svFile, selector) + return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file didn't select any of the installed toolchains. You can install one with `swiftly install \(selector.description)`."))) } - return (selectedToolchain, svFile, selector) + return (selectedToolchain, .swiftVersionFile(svFile, nil)) } cwd = cwd.deletingLastPathComponent() } } - if let inUse = config.inUse { - return (inUse, nil, nil) - } - - return nil + return (config.inUse, .globalDefault) } From 66dd45985ffc88adf153ae0bf841ffd4ac62e90c Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Thu, 5 Sep 2024 15:36:59 -0400 Subject: [PATCH 09/26] Update the documentation --- .../SwiftlyDocs.docc/swiftly-cli-reference.md | 20 ++++++++--- .../GenerateDocsReference.swift | 33 +++++++++++-------- 2 files changed, 34 insertions(+), 19 deletions(-) diff --git a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md index 74af4da5..9ba8b475 100644 --- a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md +++ b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md @@ -23,7 +23,7 @@ swiftly [--version] [--help] Install a new toolchain. ``` -swiftly install [--use] [--token=] [--verify] [--post-install-file=] [--version] [--help] +swiftly install [--use] [--token=] [--verify|no-verify] [--post-install-file=] [--version] [--help] ``` **version:** @@ -67,7 +67,7 @@ This is useful to avoid GitHub's low rate limits. If an installation fails with an "unauthorized" status code, it likely means the rate limit has been hit. -**--verify:** +**--verify|no-verify:** *Verify the toolchain's PGP signature before proceeding with installation.* @@ -142,9 +142,19 @@ Note that listing available snapshots before 6.0 is unsupported. Set the active toolchain. If no toolchain is provided, print the currently in-use toolchain, if any. ``` -swiftly use [] [--version] [--help] +swiftly use [--print-location] [--global-default] [] [--version] [--help] ``` +**--print-location:** + +*Print the location of the in-use toolchain. This is valid only when there is no toolchain argument.* + + +**--global-default:** + +*Use the global default, ignoring any .swift-version files.* + + **toolchain:** *The toolchain to use.* @@ -293,7 +303,7 @@ The installed snapshots for a given devlopment branch can be listed by specifyin Update an installed toolchain to a newer version. ``` -swiftly update [] [--assume-yes] [--verify] [--post-install-file=] [--version] [--help] +swiftly update [] [--assume-yes] [--verify|no-verify] [--post-install-file=] [--version] [--help] ``` **toolchain:** @@ -339,7 +349,7 @@ A specific snapshot toolchain can be updated by including the date: *Disable confirmation prompts by assuming 'yes'* -**--verify:** +**--verify|no-verify:** *Verify the toolchain's PGP signature before proceeding with installation.* diff --git a/Tools/generate-docs-reference/GenerateDocsReference.swift b/Tools/generate-docs-reference/GenerateDocsReference.swift index 77cc2cde..539a3ab4 100644 --- a/Tools/generate-docs-reference/GenerateDocsReference.swift +++ b/Tools/generate-docs-reference/GenerateDocsReference.swift @@ -142,11 +142,14 @@ extension ArgumentInfoV0 { return "" } - let name: String - if let preferred = self.preferredName { - name = preferred.name + let names: [String] + + if let myNames = self.names { + names = myNames.filter { $0.kind == .long }.map(\.name) + } else if let preferred = self.preferredName { + names = [preferred.name] } else if let value = self.valueName { - name = value + names = [value] } else { return "" } @@ -157,11 +160,11 @@ extension ArgumentInfoV0 { switch self.kind { case .positional: - "<\(name)>" + "<\(names.joined(separator: "|"))>" case .option: - "--\(name)=<\(self.valueName ?? "")>" + "--\(names.joined(separator: "|"))=<\(self.valueName ?? "")>" case .flag: - "--\(name)" + "--\(names.joined(separator: "|"))" } if self.isRepeating { @@ -176,11 +179,13 @@ extension ArgumentInfoV0 { } public func identity() -> String { - let name: String - if let preferred = self.preferredName { - name = preferred.name + let names: [String] + if let myNames = self.names { + names = myNames.filter { $0.kind == .long }.map(\.name) + } else if let preferred = self.preferredName { + names = [preferred.name] } else if let value = self.valueName { - name = value + names = [value] } else { return "" } @@ -191,11 +196,11 @@ extension ArgumentInfoV0 { switch self.kind { case .positional: - "\(name)" + "\(names.joined(separator: "|"))" case .option: - "--\(name)=\\<\(self.valueName ?? "")\\>" + "--\(names.joined(separator: "|"))=\\<\(self.valueName ?? "")\\>" case .flag: - "--\(name)" + "--\(names.joined(separator: "|"))" } return inner From 2df7357799b9140324c2714bc44588cfe9a49964 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Fri, 6 Sep 2024 15:56:04 -0400 Subject: [PATCH 10/26] Create a swiftly run command Provide a run command that allows arbitrary commands to be run in the context of the selected toolchain. Provide a one-off selection mechanism with a special syntax on the run command. --- .../SwiftlyDocs.docc/swiftly-cli-reference.md | 58 +++++++ Sources/Swiftly/Install.swift | 4 +- Sources/Swiftly/Proxy.swift | 6 +- Sources/Swiftly/Run.swift | 146 ++++++++++++++++++ Sources/Swiftly/Swiftly.swift | 1 + Sources/Swiftly/Use.swift | 21 ++- Sources/SwiftlyCore/Platform.swift | 71 ++++++++- 7 files changed, 293 insertions(+), 14 deletions(-) create mode 100644 Sources/Swiftly/Run.swift diff --git a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md index 9ba8b475..106202f3 100644 --- a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md +++ b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md @@ -434,3 +434,61 @@ swiftly self-update [--version] [--help] +## run + +Run a command while proxying to the selected toolchain commands. + +``` +swiftly run ... [--version] [--help] +``` + +**command:** + +*Run a command while proxying to the selected toolchain commands.* + + +Run a command with a selected toolchain, so that all toolchain commands are become the default added to the system path and other common environment variables. + +You can run one of the usual toolchain commands directly: + + $ swiftly run swift build + +Or you can run another program (or script) that runs one or more toolchain commands: + + $ swiftly run make # Builds targets using clang/swiftc + $ swiftly run ./build-things.sh # Script invokes 'swift build' to create certain product binaries + +Toolchain selection is determined by swift version files `.swift-version`, with a default global as the fallback. See the `swiftly use` command for more details. + +You can also override the selection mechanisms temporarily for the duration of the command using a special syntax. An argument prefixed with a '+' will be treated as the selector. + + $ swiftly run swift build +latest + $ swiftly run swift build +5.10.1 + +The first command builds the swift package with the latest toolchain and the second selects the 5.10.1 toolchain. Note that if these aren't installed then run will fail with an error message. You can pre-install the toolchain using `swiftly install ` to ensure success. There is also a `+install` argument that will automatically download and install the toolchain if necessary. + + $ swiftly run swift build +latest +install + +If the command that you are running needs the arguments with the '+' prefixes then you can escape it by doubling the '++'. + + $ swiftly run ./myscript.sh ++abcde + +The script will receive the argument as '+abcde'. If there are multiple arguments with the '+' prefix that should be escaped you can disable the selection using a '++++' argument, which turns off any selector argument processing for subsequent arguments. This is anologous to the '--' that turns off flag and option processing for subsequent arguments in many argument parsers. + + $ swiftly run ./myscript.sh ++++ +abcde +xyz + +The script will receive the argument '+abcde' followed by '+xyz'. + + +**--version:** + +*Show the version.* + + +**--help:** + +*Show help information.* + + + + diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index 4b097997..57b0806f 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -101,7 +101,7 @@ struct Install: SwiftlyCommand { } } - internal static func execute( + public static func execute( version: ToolchainVersion, _ config: inout Config, useInstalledToolchain: Bool, @@ -227,7 +227,7 @@ struct Install: SwiftlyCommand { /// Utilize the GitHub API along with the provided selector to select a toolchain for install. /// TODO: update this to use an official swift.org API - static func resolve(config: Config, selector: ToolchainSelector) async throws -> ToolchainVersion { + public static func resolve(config: Config, selector: ToolchainSelector) async throws -> ToolchainVersion { switch selector { case .latest: SwiftlyCore.print("Fetching the latest stable Swift release...") diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift index 45b3dcea..3deab60e 100644 --- a/Sources/Swiftly/Proxy.swift +++ b/Sources/Swiftly/Proxy.swift @@ -35,9 +35,9 @@ public enum Proxy { return } - let config = try Config.load() + var config = try Config.load() - let (toolchain, result) = selectToolchain(config: config) + let (toolchain, result) = try await selectToolchain(config: &config) // Abort on any errors relating to swift version files if case let .swiftVersionFile(_, error) = result, let error = error { @@ -49,6 +49,8 @@ public enum Proxy { } try await Swiftly.currentPlatform.proxy(toolchain, binName, Array(CommandLine.arguments[1...])) + } catch let terminated as RunProgramError { + exit(terminated.exitCode) } catch { SwiftlyCore.print("\(error)") exit(1) diff --git a/Sources/Swiftly/Run.swift b/Sources/Swiftly/Run.swift new file mode 100644 index 00000000..8aaeac1e --- /dev/null +++ b/Sources/Swiftly/Run.swift @@ -0,0 +1,146 @@ +import ArgumentParser +import Foundation +import SwiftlyCore + +internal struct Run: SwiftlyCommand { + public static var configuration = CommandConfiguration( + abstract: "Run a command while proxying to the selected toolchain commands." + ) + + @Argument(parsing: .captureForPassthrough, help: ArgumentHelp( + "Run a command while proxying to the selected toolchain commands.", + discussion: """ + + Run a command with a selected toolchain, so that all toolchain commands \ + are become the default added to the system path and other common environment \ + variables. + + You can run one of the usual toolchain commands directly: + + $ swiftly run swift build + + Or you can run another program (or script) that runs one or more toolchain commands: + + $ swiftly run make # Builds targets using clang/swiftc + $ swiftly run ./build-things.sh # Script invokes 'swift build' to create certain product binaries + + Toolchain selection is determined by swift version files `.swift-version`, with a default global \ + as the fallback. See the `swiftly use` command for more details. + + You can also override the selection mechanisms temporarily for the duration of the command using \ + a special syntax. An argument prefixed with a '+' will be treated as the selector. + + $ swiftly run swift build +latest + $ swiftly run swift build +5.10.1 + + The first command builds the swift package with the latest toolchain and the second selects the \ + 5.10.1 toolchain. Note that if these aren't installed then run will fail with an error message. \ + You can pre-install the toolchain using `swiftly install ` to ensure success. There is \ + also a `+install` argument that will automatically download and install the toolchain if necessary. + + $ swiftly run swift build +latest +install + + If the command that you are running needs the arguments with the '+' prefixes then you can escape \ + it by doubling the '++'. + + $ swiftly run ./myscript.sh ++abcde + + The script will receive the argument as '+abcde'. If there are multiple arguments with the '+' prefix \ + that should be escaped you can disable the selection using a '++++' argument, which turns off any \ + selector argument processing for subsequent arguments. This is anologous to the '--' that turns off \ + flag and option processing for subsequent arguments in many argument parsers. + + $ swiftly run ./myscript.sh ++++ +abcde +xyz + + The script will receive the argument '+abcde' followed by '+xyz'. + """ + )) + var command: [String] + + internal mutating func run() async throws { + try validateSwiftly() + + guard self.command.count > 0 else { + throw Error(message: "Provide at least one command to run") + } + + var escapedCommand: [String] = [] + var selector: ToolchainSelector? + var install = false + var disableEscaping = false + for c in self.command { + if !disableEscaping && c == "++++" { + disableEscaping = true + continue + } + + if !disableEscaping && c.hasPrefix("++") { + escapedCommand.append("+\(String(c.dropFirst(2)))") + continue + } + + if !disableEscaping && c == "+install" { + install = true + continue + } + + if !disableEscaping && c.hasPrefix("+") { + selector = try ToolchainSelector(parsing: String(c.dropFirst())) + continue + } + + escapedCommand.append(c) + } + + var config = try Config.load() + + let toolchain: ToolchainVersion? + + if let selector = selector { + if install { + let version = try await Install.resolve(config: config, selector: selector) + let postInstallScript = try await Install.execute(version: version, &config, useInstalledToolchain: false, verifySignature: true) + if let postInstallScript = postInstallScript { + throw Error(message: """ + + There are some system dependencies that should be installed before using this toolchain. + You can run the following script as the system administrator (e.g. root) to prepare + your system: + + \(postInstallScript) + """) + } + + toolchain = version + } else { + let matchedToolchain = config.listInstalledToolchains(selector: selector).max() + guard let matchedToolchain = matchedToolchain else { + throw Error(message: "The selected toolchain \(selector.description) didn't matched any of the installed toolchains. You can install it by adding '+install' to your command, or `swiftly install \(selector.description)`") + } + + toolchain = matchedToolchain + } + } else { + let (version, result) = try await selectToolchain(config: &config, install: install) + + // Abort on any errors relating to swift version files + if case let .swiftVersionFile(_, error) = result, let error = error { + throw error + } + + toolchain = version + } + + guard let toolchain = toolchain else { + throw Error(message: "No swift toolchain could be selected from either from a .swift-version file, or the default. You can try using `swiftly install ` to install one.") + } + + do { + try await Swiftly.currentPlatform.proxy(toolchain, escapedCommand[0], [String](escapedCommand[1...])) + } catch let terminated as RunProgramError { + Foundation.exit(terminated.exitCode) + } catch { + throw error + } + } +} diff --git a/Sources/Swiftly/Swiftly.swift b/Sources/Swiftly/Swiftly.swift index 7d9789e0..1652c9a2 100644 --- a/Sources/Swiftly/Swiftly.swift +++ b/Sources/Swiftly/Swiftly.swift @@ -29,6 +29,7 @@ public struct Swiftly: SwiftlyCommand { Update.self, Init.self, SelfUpdate.self, + Run.self, ] ) diff --git a/Sources/Swiftly/Use.swift b/Sources/Swiftly/Use.swift index ced9e4c9..38066c65 100644 --- a/Sources/Swiftly/Use.swift +++ b/Sources/Swiftly/Use.swift @@ -55,7 +55,7 @@ internal struct Use: SwiftlyCommand { // This is the bare use command where we print the selected toolchain version (or the path to it) guard let toolchain = self.toolchain else { - let (selectedVersion, result) = selectToolchain(config: config, globalDefault: self.globalDefault) + let (selectedVersion, result) = try await selectToolchain(config: &config, globalDefault: self.globalDefault) // Abort on any errors with the swift version files if case let .swiftVersionFile(_, error) = result, let error = error { @@ -103,7 +103,7 @@ internal struct Use: SwiftlyCommand { /// Use a toolchain. This method can modify and save the input config. internal static func execute(_ toolchain: ToolchainVersion, _ globalDefault: Bool, _ config: inout Config) async throws { - let (selectedVersion, result) = selectToolchain(config: config, globalDefault: globalDefault) + let (selectedVersion, result) = try await selectToolchain(config: &config, globalDefault: globalDefault) if let selectedVersion = selectedVersion { guard selectedVersion != toolchain else { @@ -175,7 +175,7 @@ public enum ToolchainSelectionResult { /// If such a case happens then the toolchain version in the tuple will be nil, but the /// result will be .swiftVersionFile and a detailed error about the problem. This error /// can be thrown by the client, or ignored. -public func selectToolchain(config: Config, globalDefault: Bool = false) -> (ToolchainVersion?, ToolchainSelectionResult) { +public func selectToolchain(config: inout Config, globalDefault: Bool = false, install: Bool = false) async throws -> (ToolchainVersion?, ToolchainSelectionResult) { if !globalDefault { var cwd = URL(fileURLWithPath: FileManager.default.currentDirectoryPath) @@ -209,6 +209,21 @@ public func selectToolchain(config: Config, globalDefault: Bool = false) -> (Too return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file is malformed: \(svFile)"))) } + if install { + let version = try await Install.resolve(config: config, selector: selector) + let postInstallScript = try await Install.execute(version: version, &config, useInstalledToolchain: false, verifySignature: true) + if let postInstallScript = postInstallScript { + throw Error(message: """ + + There are some system dependencies that should be installed before using this toolchain. + You can run the following script as the system administrator (e.g. root) to prepare + your system: + + \(postInstallScript) + """) + } + } + guard let selectedToolchain = config.listInstalledToolchains(selector: selector).max() else { return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file didn't select any of the installed toolchains. You can install one with `swiftly install \(selector.description)`."))) } diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index b7c2e99e..1ca5fe7e 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -29,6 +29,11 @@ public struct PlatformDefinition: Codable, Equatable { public static let amazonlinux2 = PlatformDefinition(name: "amazonlinux2", nameFull: "amazonlinux2", namePretty: "Amazon Linux 2") } +public struct RunProgramError: Swift.Error { + public let exitCode: Int32 + public let program: String +} + public protocol Platform { /// The platform-specific location on disk where applications are /// supposed to store their custom data. @@ -124,20 +129,62 @@ extension Platform { } #if os(macOS) || os(Linux) + /// Proxy the invocation of the provided command to the chosen toolchain. + /// + /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with + /// the exit code and program information. + /// public func proxy(_ toolchain: ToolchainVersion, _ command: String, _ arguments: [String]) async throws { - let cmd = self.findToolchainLocation(toolchain).appendingPathComponent("usr/bin/\(command)") - try self.runProgram([cmd.path] + arguments) + // The toolchain goes to the beginning of the path, and the SWIFTLY_BIN_DIR is removed from it + let tcPath = self.findToolchainLocation(toolchain).appendingPathComponent("usr/bin") + var newEnv = ProcessInfo.processInfo.environment + + // Prevent circularities with a memento environment variable + guard newEnv["SWIFTLY_PROXY_IN_PROGRESS"] == nil else { + throw Error(message: "Circular swiftly proxy invocation") + } + newEnv["SWIFTLY_PROXY_IN_PROGRESS"] = "1" + + var newPath = newEnv["PATH"] ?? "" + if !newPath.hasPrefix(tcPath.path + ":") { + newPath = ([tcPath.path] + newPath.split(separator: ":").map { String($0) }.filter { $0 != swiftlyBinDir.path }).joined(separator: ":") + } + newEnv["PATH"] = newPath + + // Remove traces of swiftly environment variables + newEnv.removeValue(forKey: "SWIFTLY_BIN_DIR") + newEnv.removeValue(forKey: "SWIFTLY_HOME_DIR") + + // Add certain common environment variables that can be used to proxy to the toolchain + newEnv["CC"] = tcPath.appendingPathComponent("clang").path + newEnv["CXX"] = tcPath.appendingPathComponent("clang++").path + + try self.runProgram([command] + arguments, env: newEnv) } - public func runProgram(_ args: String..., quiet: Bool = false) throws { - try self.runProgram([String](args), quiet: quiet) + /// Run a program. + /// + /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with + /// the exit code and program information. + /// + public func runProgram(_ args: String..., quiet: Bool = false, env: [String: String]? = nil) throws { + try self.runProgram([String](args), quiet: quiet, env: env) } - public func runProgram(_ args: [String], quiet: Bool = false) throws { + /// Run a program. + /// + /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with + /// the exit code and program information. + /// + public func runProgram(_ args: [String], quiet: Bool = false, env: [String: String]? = nil) throws { let process = Process() process.executableURL = URL(fileURLWithPath: "/usr/bin/env") process.arguments = args + if let env = env { + process.environment = env + } + if quiet { process.standardOutput = nil process.standardError = nil @@ -152,14 +199,24 @@ extension Platform { process.waitUntilExit() guard process.terminationStatus == 0 else { - throw Error(message: "\(args.first!) exited with non-zero status: \(process.terminationStatus)") + throw RunProgramError(exitCode: process.terminationStatus, program: args.first!) } } + /// Run a program and capture its output. + /// + /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with + /// the exit code and program information. + /// public func runProgramOutput(_ program: String, _ args: String...) async throws -> String? { try await self.runProgramOutput(program, [String](args)) } + /// Run a program and capture its output. + /// + /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with + /// the exit code and program information. + /// public func runProgramOutput(_ program: String, _ args: [String]) async throws -> String? { let process = Process() process.executableURL = URL(fileURLWithPath: "/usr/bin/env") @@ -181,7 +238,7 @@ extension Platform { process.waitUntilExit() guard process.terminationStatus == 0 else { - throw Error(message: "\(args.first!) exited with non-zero status: \(process.terminationStatus)") + throw RunProgramError(exitCode: process.terminationStatus, program: args.first!) } if let outData = outData { From 8976bba8a60476524b30bc4b09e5a3a910e3c396 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sat, 7 Sep 2024 07:14:29 -0400 Subject: [PATCH 11/26] Fix empty command case with a single ++++ --- Sources/Swiftly/Install.swift | 2 +- Sources/Swiftly/Run.swift | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index 57b0806f..a15a931e 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -108,7 +108,7 @@ struct Install: SwiftlyCommand { verifySignature: Bool ) async throws -> String? { guard !config.installedToolchains.contains(version) else { - SwiftlyCore.print("\(version) is already installed, exiting.") + SwiftlyCore.print("\(version) is already installed.") return nil } diff --git a/Sources/Swiftly/Run.swift b/Sources/Swiftly/Run.swift index 8aaeac1e..7b31d6b0 100644 --- a/Sources/Swiftly/Run.swift +++ b/Sources/Swiftly/Run.swift @@ -60,10 +60,6 @@ internal struct Run: SwiftlyCommand { internal mutating func run() async throws { try validateSwiftly() - guard self.command.count > 0 else { - throw Error(message: "Provide at least one command to run") - } - var escapedCommand: [String] = [] var selector: ToolchainSelector? var install = false @@ -92,6 +88,10 @@ internal struct Run: SwiftlyCommand { escapedCommand.append(c) } + guard escapedCommand.count > 0 else { + throw Error(message: "Provide at least one command to run") + } + var config = try Config.load() let toolchain: ToolchainVersion? From 3caab6679634299a841221a6129f6ee39b743785 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Mon, 9 Sep 2024 11:14:05 -0400 Subject: [PATCH 12/26] Write run command and proxy tests --- Sources/Swiftly/Run.swift | 87 +++++++++++++++---------- Sources/SwiftlyCore/Platform.swift | 45 ++++++++----- Tests/SwiftlyTests/RunTests.swift | 101 +++++++++++++++++++++++++++++ 3 files changed, 181 insertions(+), 52 deletions(-) create mode 100644 Tests/SwiftlyTests/RunTests.swift diff --git a/Sources/Swiftly/Run.swift b/Sources/Swiftly/Run.swift index 7b31d6b0..12df7634 100644 --- a/Sources/Swiftly/Run.swift +++ b/Sources/Swiftly/Run.swift @@ -46,11 +46,11 @@ internal struct Run: SwiftlyCommand { $ swiftly run ./myscript.sh ++abcde The script will receive the argument as '+abcde'. If there are multiple arguments with the '+' prefix \ - that should be escaped you can disable the selection using a '++++' argument, which turns off any \ + that should be escaped you can disable the selection using a '++' argument, which turns off any \ selector argument processing for subsequent arguments. This is anologous to the '--' that turns off \ flag and option processing for subsequent arguments in many argument parsers. - $ swiftly run ./myscript.sh ++++ +abcde +xyz + $ swiftly run ./myscript.sh ++ +abcde +xyz The script will receive the argument '+abcde' followed by '+xyz'. """ @@ -60,40 +60,10 @@ internal struct Run: SwiftlyCommand { internal mutating func run() async throws { try validateSwiftly() - var escapedCommand: [String] = [] - var selector: ToolchainSelector? - var install = false - var disableEscaping = false - for c in self.command { - if !disableEscaping && c == "++++" { - disableEscaping = true - continue - } - - if !disableEscaping && c.hasPrefix("++") { - escapedCommand.append("+\(String(c.dropFirst(2)))") - continue - } - - if !disableEscaping && c == "+install" { - install = true - continue - } - - if !disableEscaping && c.hasPrefix("+") { - selector = try ToolchainSelector(parsing: String(c.dropFirst())) - continue - } - - escapedCommand.append(c) - } - - guard escapedCommand.count > 0 else { - throw Error(message: "Provide at least one command to run") - } - var config = try Config.load() + let (command, selector, install) = try extractProxyArguments(command: self.command) + let toolchain: ToolchainVersion? if let selector = selector { @@ -115,7 +85,7 @@ internal struct Run: SwiftlyCommand { } else { let matchedToolchain = config.listInstalledToolchains(selector: selector).max() guard let matchedToolchain = matchedToolchain else { - throw Error(message: "The selected toolchain \(selector.description) didn't matched any of the installed toolchains. You can install it by adding '+install' to your command, or `swiftly install \(selector.description)`") + throw Error(message: "The selected toolchain \(selector.description) didn't match any of the installed toolchains. You can install it by adding '+install' to your command, or `swiftly install \(selector.description)`") } toolchain = matchedToolchain @@ -136,7 +106,16 @@ internal struct Run: SwiftlyCommand { } do { - try await Swiftly.currentPlatform.proxy(toolchain, escapedCommand[0], [String](escapedCommand[1...])) + if let outputHandler = SwiftlyCore.outputHandler { + if let output = try await Swiftly.currentPlatform.proxyOutput(toolchain, command[0], [String](command[1...])) { + for line in output.split(separator: "\n") { + outputHandler.handleOutputLine(String(line)) + } + } + return + } + + try await Swiftly.currentPlatform.proxy(toolchain, command[0], [String](command[1...])) } catch let terminated as RunProgramError { Foundation.exit(terminated.exitCode) } catch { @@ -144,3 +123,39 @@ internal struct Run: SwiftlyCommand { } } } + +public func extractProxyArguments(command: [String]) throws -> (command: [String], selector: ToolchainSelector?, install: Bool) { + var args: (command: [String], selector: ToolchainSelector?, install: Bool) = (command: [], nil, false) + + var disableEscaping = false + + for c in command { + if !disableEscaping && c == "++" { + disableEscaping = true + continue + } + + if !disableEscaping && c.hasPrefix("++") { + args.command.append("+\(String(c.dropFirst(2)))") + continue + } + + if !disableEscaping && c == "+install" { + args.install = true + continue + } + + if !disableEscaping && c.hasPrefix("+") { + args.selector = try ToolchainSelector(parsing: String(c.dropFirst())) + continue + } + + args.command.append(c) + } + + guard args.command.count > 0 else { + throw Error(message: "Provide at least one command to run.") + } + + return args +} diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index 1ca5fe7e..9042a8c4 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -129,13 +129,7 @@ extension Platform { } #if os(macOS) || os(Linux) - /// Proxy the invocation of the provided command to the chosen toolchain. - /// - /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with - /// the exit code and program information. - /// - public func proxy(_ toolchain: ToolchainVersion, _ command: String, _ arguments: [String]) async throws { - // The toolchain goes to the beginning of the path, and the SWIFTLY_BIN_DIR is removed from it + internal func proxyEnv(_ toolchain: ToolchainVersion) throws -> [String: String] { let tcPath = self.findToolchainLocation(toolchain).appendingPathComponent("usr/bin") var newEnv = ProcessInfo.processInfo.environment @@ -145,21 +139,36 @@ extension Platform { } newEnv["SWIFTLY_PROXY_IN_PROGRESS"] = "1" + // The toolchain goes to the beginning of the PATH var newPath = newEnv["PATH"] ?? "" if !newPath.hasPrefix(tcPath.path + ":") { - newPath = ([tcPath.path] + newPath.split(separator: ":").map { String($0) }.filter { $0 != swiftlyBinDir.path }).joined(separator: ":") + newPath = ([tcPath.path] + newPath.split(separator: ":").map { String($0) }).joined(separator: ":") } newEnv["PATH"] = newPath - // Remove traces of swiftly environment variables - newEnv.removeValue(forKey: "SWIFTLY_BIN_DIR") - newEnv.removeValue(forKey: "SWIFTLY_HOME_DIR") - // Add certain common environment variables that can be used to proxy to the toolchain newEnv["CC"] = tcPath.appendingPathComponent("clang").path newEnv["CXX"] = tcPath.appendingPathComponent("clang++").path - try self.runProgram([command] + arguments, env: newEnv) + return newEnv + } + + /// Proxy the invocation of the provided command to the chosen toolchain. + /// + /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with + /// the exit code and program information. + /// + public func proxy(_ toolchain: ToolchainVersion, _ command: String, _ arguments: [String]) async throws { + try self.runProgram([command] + arguments, env: self.proxyEnv(toolchain)) + } + + /// Proxy the invocation of the provided command to the chosen toolchain and capture the output. + /// + /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with + /// the exit code and program information. + /// + public func proxyOutput(_ toolchain: ToolchainVersion, _ command: String, _ arguments: [String]) async throws -> String? { + try await self.runProgramOutput(command, arguments, env: self.proxyEnv(toolchain)) } /// Run a program. @@ -208,8 +217,8 @@ extension Platform { /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with /// the exit code and program information. /// - public func runProgramOutput(_ program: String, _ args: String...) async throws -> String? { - try await self.runProgramOutput(program, [String](args)) + public func runProgramOutput(_ program: String, _ args: String..., env: [String: String]? = nil) async throws -> String? { + try await self.runProgramOutput(program, [String](args), env: env) } /// Run a program and capture its output. @@ -217,11 +226,15 @@ extension Platform { /// In the case where the command exit with a non-zero exit code a RunProgramError is thrown with /// the exit code and program information. /// - public func runProgramOutput(_ program: String, _ args: [String]) async throws -> String? { + public func runProgramOutput(_ program: String, _ args: [String], env: [String: String]? = nil) async throws -> String? { let process = Process() process.executableURL = URL(fileURLWithPath: "/usr/bin/env") process.arguments = [program] + args + if let env = env { + process.environment = env + } + let outPipe = Pipe() process.standardInput = FileHandle.nullDevice process.standardError = FileHandle.nullDevice diff --git a/Tests/SwiftlyTests/RunTests.swift b/Tests/SwiftlyTests/RunTests.swift new file mode 100644 index 00000000..ce462700 --- /dev/null +++ b/Tests/SwiftlyTests/RunTests.swift @@ -0,0 +1,101 @@ +import Foundation +@testable import Swiftly +@testable import SwiftlyCore +import XCTest + +final class RunTests: SwiftlyTests { + static let homeName = "runTests" + + /// Tests that the `run` command can switch between installed toolchains. + func testRunSelection() async throws { + try await self.withMockedHome(homeName: Self.homeName, toolchains: Self.allToolchains) { + // GIVEN: a set of installed toolchains + // WHEN: invoking the run command with a selector argument for that toolchain + var run = try self.parseCommand(Run.self, ["run", "swift", "--version", "+\(Self.newStable.name)"]) + var output = try await run.runWithMockedIO() + // THEN: the output confirms that it ran with the selected toolchain + XCTAssert(output.contains(Self.newStable.name)) + + // GIVEN: a set of installed toolchains and one is selected with a .swift-version file + let versionFile = URL(fileURLWithPath: FileManager.default.currentDirectoryPath).appendingPathComponent(".swift-version") + try Self.oldStable.name.write(to: versionFile, atomically: true, encoding: .utf8) + // WHEN: invoking the run command without any selector arguments for toolchains + run = try self.parseCommand(Run.self, ["run", "swift", "--version"]) + output = try await run.runWithMockedIO() + // THEN: the output confirms that it ran with the selected toolchain + XCTAssert(output.contains(Self.oldStable.name)) + + // GIVEN: a set of installed toolchains + // WHEN: invoking the run command with a selector argument for a toolchain that isn't installed + run = try self.parseCommand(Run.self, ["run", "swift", "+1.2.3", "--version"]) + do { + try await run.run() + XCTAssert(false) + } catch let e as Error { + XCTAssert(e.message.contains("didn't match any of the installed toolchains")) + } + // THEN: an error is shown because there is no matching toolchain that is installed + } + } + + /// Tests the `run` command verifying that the environment is as expected + func testRunEnvironment() async throws { + try await self.withMockedHome(homeName: Self.homeName, toolchains: Self.allToolchains) { + // The toolchains directory should be the fist entry on the path + var run = try self.parseCommand(Run.self, ["run", try await Swiftly.currentPlatform.getShell(), "-c", "echo $PATH"]) + var output = try await run.runWithMockedIO() + XCTAssert(output[0].contains(Swiftly.currentPlatform.swiftlyToolchainsDir.path)) + + // The CC and CXX variables should be set to clang/clang++ in the toolchains + run = try self.parseCommand(Run.self, ["run", try await Swiftly.currentPlatform.getShell(), "-c", "echo $CC; echo $CXX"]) + output = try await run.runWithMockedIO() + XCTAssert(output[0].hasPrefix(Swiftly.currentPlatform.swiftlyToolchainsDir.path)) + XCTAssert(output[0].hasSuffix("clang")) + XCTAssert(output[1].hasPrefix(Swiftly.currentPlatform.swiftlyToolchainsDir.path)) + XCTAssert(output[1].hasSuffix("clang++")) + } + } + + /// Tests the extraction of proxy arguments from the run command arguments. + func testExtractProxyArguments() throws { + var (command, selector, install) = try extractProxyArguments(command: ["swift", "build"]) + XCTAssertEqual(["swift", "build"], command) + XCTAssertEqual(false, install) + XCTAssertEqual(nil, selector) + + (command, selector, install) = try extractProxyArguments(command: ["swift", "+1.2.3", "build"]) + XCTAssertEqual(["swift", "build"], command) + XCTAssertEqual(false, install) + XCTAssertEqual(try! ToolchainSelector(parsing: "1.2.3"), selector) + + (command, selector, install) = try extractProxyArguments(command: ["swift", "build", "+latest"]) + XCTAssertEqual(["swift", "build"], command) + XCTAssertEqual(false, install) + XCTAssertEqual(try! ToolchainSelector(parsing: "latest"), selector) + + (command, selector, install) = try extractProxyArguments(command: ["+5.6", "swift", "build"]) + XCTAssertEqual(["swift", "build"], command) + XCTAssertEqual(false, install) + XCTAssertEqual(try! ToolchainSelector(parsing: "5.6"), selector) + + (command, selector, install) = try extractProxyArguments(command: ["swift", "++1.2.3", "build"]) + XCTAssertEqual(["swift", "+1.2.3", "build"], command) + XCTAssertEqual(false, install) + XCTAssertEqual(nil, selector) + + (command, selector, install) = try extractProxyArguments(command: ["swift", "++", "+1.2.3", "build"]) + XCTAssertEqual(["swift", "+1.2.3", "build"], command) + XCTAssertEqual(false, install) + XCTAssertEqual(nil, selector) + + do { + let _ = try extractProxyArguments(command: ["+1.2.3"]) + XCTAssert(false) + } catch {} + + do { + let _ = try extractProxyArguments(command: []) + XCTAssert(false) + } catch {} + } +} From 31f43272df012809020c4d0544139834613d65c7 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Mon, 9 Sep 2024 11:19:52 -0400 Subject: [PATCH 13/26] Regenerate the cli reference documentation --- Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md index 106202f3..9edcdc83 100644 --- a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md +++ b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md @@ -473,9 +473,9 @@ If the command that you are running needs the arguments with the '+' prefixes th $ swiftly run ./myscript.sh ++abcde -The script will receive the argument as '+abcde'. If there are multiple arguments with the '+' prefix that should be escaped you can disable the selection using a '++++' argument, which turns off any selector argument processing for subsequent arguments. This is anologous to the '--' that turns off flag and option processing for subsequent arguments in many argument parsers. +The script will receive the argument as '+abcde'. If there are multiple arguments with the '+' prefix that should be escaped you can disable the selection using a '++' argument, which turns off any selector argument processing for subsequent arguments. This is anologous to the '--' that turns off flag and option processing for subsequent arguments in many argument parsers. - $ swiftly run ./myscript.sh ++++ +abcde +xyz + $ swiftly run ./myscript.sh ++ +abcde +xyz The script will receive the argument '+abcde' followed by '+xyz'. From 16caf80730f9bbf2d73b352eef111816d2a7444c Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Mon, 9 Sep 2024 11:54:11 -0400 Subject: [PATCH 14/26] Fix design document discrepancies and add install proxy argument tests --- DESIGN.md | 2 +- Tests/SwiftlyTests/RunTests.swift | 10 ++++++++++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/DESIGN.md b/DESIGN.md index e3aa5efb..e6a814c9 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -324,7 +324,7 @@ Swiftly adjusts certain environment variables, such as prefixing the PATH to the swiftly run swift build +5.10.1 # Runs swift build with the 5.10.1 toolchain ``` -A few notes about the '+' prefix. First, if a literal '+' prefix should be sent directly to the tool as an argument then it is escaped by doubling it with '++'. An argument with only '++++' is ignored entirely, but any additional arguments are sent directly to the command without any further inspection of their prefixes. This is analogous to the special '--' token that certain argument parsers accept so that they don't interpret anything following that token as command flags or options. +A few notes about the '+' prefix. First, if a literal '+' prefix should be sent directly to the tool as an argument then it is escaped by doubling it with '++'. An argument with only '++' is ignored entirely, and any additional arguments are sent directly to the command without any further inspection of their prefixes. This is analogous to the special '--' token that certain argument parsers accept so that they don't interpret anything following that token as command flags or options. If the selected toolchain is not installed then swiftly will exit with a message indicating that you need to run `swiftly install x.y.z` to install it. However, if you enter a special `+install` token then swiftly will automatically download and install the toolchain if it isn't already present. diff --git a/Tests/SwiftlyTests/RunTests.swift b/Tests/SwiftlyTests/RunTests.swift index ce462700..7b92c38b 100644 --- a/Tests/SwiftlyTests/RunTests.swift +++ b/Tests/SwiftlyTests/RunTests.swift @@ -97,5 +97,15 @@ final class RunTests: SwiftlyTests { let _ = try extractProxyArguments(command: []) XCTAssert(false) } catch {} + + (command, selector, install) = try extractProxyArguments(command: ["swift", "+1.2.3", "+install", "build"]) + XCTAssertEqual(["swift", "build"], command) + XCTAssertEqual(true, install) + XCTAssertEqual(try! ToolchainSelector(parsing: "1.2.3"), selector) + + (command, selector, install) = try extractProxyArguments(command: ["swift", "+install", "build"]) + XCTAssertEqual(["swift", "build"], command) + XCTAssertEqual(true, install) + XCTAssertEqual(nil, selector) } } From b2aa16570bd1acf60583d612b049ed775fb7544c Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Tue, 10 Sep 2024 06:51:55 -0400 Subject: [PATCH 15/26] Update the list command to decorate default, and in-use toolchains --- Sources/Swiftly/List.swift | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/Sources/Swiftly/List.swift b/Sources/Swiftly/List.swift index 285c7933..1757532e 100644 --- a/Sources/Swiftly/List.swift +++ b/Sources/Swiftly/List.swift @@ -39,16 +39,19 @@ struct List: SwiftlyCommand { try ToolchainSelector(parsing: input) } - let config = try Config.load() + var config = try Config.load() let toolchains = config.listInstalledToolchains(selector: selector).sorted { $0 > $1 } - let activeToolchain = config.inUse + let (inUse, _) = try await selectToolchain(config: &config) let printToolchain = { (toolchain: ToolchainVersion) in var message = "\(toolchain)" - if toolchain == activeToolchain { + if let inUse = inUse, toolchain == inUse { message += " (in use)" } + if toolchain == config.inUse { + message += " (default)" + } SwiftlyCore.print(message) } From 7504dd3247a85a69664289fbd4f9fdf01f2187a3 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Tue, 10 Sep 2024 07:01:39 -0400 Subject: [PATCH 16/26] Update list tests to check for in use and default labels --- Tests/SwiftlyTests/ListTests.swift | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Tests/SwiftlyTests/ListTests.swift b/Tests/SwiftlyTests/ListTests.swift index ae2c1d37..6724ab97 100644 --- a/Tests/SwiftlyTests/ListTests.swift +++ b/Tests/SwiftlyTests/ListTests.swift @@ -128,7 +128,7 @@ final class ListTests: SwiftlyTests { let output = try await list.runWithMockedIO() let inUse = output.filter { $0.contains("in use") } - XCTAssertEqual(inUse, ["\(toolchain) (in use)"]) + XCTAssertEqual(inUse, ["\(toolchain) (in use) (default)"]) } try await self.runListTest { From 0e7b6616285379961eda6975ed7d02aa570ddb9b Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Wed, 11 Sep 2024 13:58:31 -0400 Subject: [PATCH 17/26] Make the version argument optional in the install subcommand With no arguments the install subcommand will install the currently selected toolchain from the `.swift-version` files. --- .../SwiftlyDocs.docc/swiftly-cli-reference.md | 6 +++- Sources/Swiftly/Install.swift | 28 +++++++++++++++++-- Sources/Swiftly/Proxy.swift | 2 +- Sources/Swiftly/Run.swift | 2 +- Sources/Swiftly/Use.swift | 20 ++++++------- 5 files changed, 43 insertions(+), 15 deletions(-) diff --git a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md index 9edcdc83..887435e1 100644 --- a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md +++ b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md @@ -23,7 +23,7 @@ swiftly [--version] [--help] Install a new toolchain. ``` -swiftly install [--use] [--token=] [--verify|no-verify] [--post-install-file=] [--version] [--help] +swiftly install [] [--use] [--token=] [--verify|no-verify] [--post-install-file=] [--version] [--help] ``` **version:** @@ -53,6 +53,10 @@ Likewise, the latest snapshot associated with a given development branch can be $ swiftly install 5.7-snapshot $ swiftly install main-snapshot + Install whatever toolchain is current selected, such as a .swift-version file: + + $ swiftly install + **--use:** diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index a15a931e..b9aa1fba 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -39,9 +39,13 @@ struct Install: SwiftlyCommand { $ swiftly install 5.7-snapshot $ swiftly install main-snapshot + + Install whatever toolchain is current selected, such as a .swift-version file: + + $ swiftly install """ )) - var version: String + var version: String? @Flag(name: .shortAndLong, help: "Mark the newly installed toolchain as in-use.") var use: Bool = false @@ -74,8 +78,28 @@ struct Install: SwiftlyCommand { mutating func run() async throws { try validateSwiftly() - let selector = try ToolchainSelector(parsing: self.version) var config = try Config.load() + + var selector: ToolchainSelector + + if let version = self.version { + selector = try ToolchainSelector(parsing: version) + } else { + if case let (_, result) = try await selectToolchain(config: &config), + case let .swiftVersionFile(_, sel, error) = result + { + if let sel = sel { + selector = sel + } else if let error = error { + throw error + } else { + throw Error(message: "Internal error selecting toolchain to install.") + } + } else { + throw Error(message: "There is no toolchain selected from a .swift-version file to install.") + } + } + SwiftlyCore.httpClient.githubToken = self.token let toolchainVersion = try await Self.resolve(config: config, selector: selector) let postInstallScript = try await Self.execute( diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift index 3deab60e..3289a8ab 100644 --- a/Sources/Swiftly/Proxy.swift +++ b/Sources/Swiftly/Proxy.swift @@ -40,7 +40,7 @@ public enum Proxy { let (toolchain, result) = try await selectToolchain(config: &config) // Abort on any errors relating to swift version files - if case let .swiftVersionFile(_, error) = result, let error = error { + if case let .swiftVersionFile(_, _, error) = result, let error = error { throw error } diff --git a/Sources/Swiftly/Run.swift b/Sources/Swiftly/Run.swift index 12df7634..f165b39b 100644 --- a/Sources/Swiftly/Run.swift +++ b/Sources/Swiftly/Run.swift @@ -94,7 +94,7 @@ internal struct Run: SwiftlyCommand { let (version, result) = try await selectToolchain(config: &config, install: install) // Abort on any errors relating to swift version files - if case let .swiftVersionFile(_, error) = result, let error = error { + if case let .swiftVersionFile(_, _, error) = result, let error = error { throw error } diff --git a/Sources/Swiftly/Use.swift b/Sources/Swiftly/Use.swift index 38066c65..d77aea57 100644 --- a/Sources/Swiftly/Use.swift +++ b/Sources/Swiftly/Use.swift @@ -58,7 +58,7 @@ internal struct Use: SwiftlyCommand { let (selectedVersion, result) = try await selectToolchain(config: &config, globalDefault: self.globalDefault) // Abort on any errors with the swift version files - if case let .swiftVersionFile(_, error) = result, let error = error { + if case let .swiftVersionFile(_, _, error) = result, let error = error { throw error } @@ -76,7 +76,7 @@ internal struct Use: SwiftlyCommand { var message = "\(selectedVersion)" switch result { - case let .swiftVersionFile(versionFile, _): + case let .swiftVersionFile(versionFile, _, _): message += " (\(versionFile.path))" case .globalDefault: message += " (default)" @@ -112,7 +112,7 @@ internal struct Use: SwiftlyCommand { } } - if case let .swiftVersionFile(versionFile, _) = result { + if case let .swiftVersionFile(versionFile, _, _) = result { // We don't care in this case if there were any problems with the swift version files, just overwrite it with the new value try toolchain.name.write(to: versionFile, atomically: true, encoding: .utf8) } else if let newVersionFile = findNewVersionFile(), !globalDefault { @@ -153,7 +153,7 @@ internal struct Use: SwiftlyCommand { public enum ToolchainSelectionResult { case globalDefault - case swiftVersionFile(URL, Error?) + case swiftVersionFile(URL, ToolchainSelector?, Error?) } /// Returns the currently selected swift toolchain, if any, with details of the selection. @@ -190,11 +190,11 @@ public func selectToolchain(config: inout Config, globalDefault: Bool = false, i let contents = try? String(contentsOf: svFile, encoding: .utf8) guard let contents = contents else { - return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file could not be read: \(svFile)"))) + return (nil, .swiftVersionFile(svFile, nil, Error(message: "The swift version file could not be read: \(svFile)"))) } guard !contents.isEmpty else { - return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file is empty: \(svFile)"))) + return (nil, .swiftVersionFile(svFile, nil, Error(message: "The swift version file is empty: \(svFile)"))) } let selectorString = contents.replacingOccurrences(of: "\n", with: "").replacingOccurrences(of: "\r", with: "") @@ -202,11 +202,11 @@ public func selectToolchain(config: inout Config, globalDefault: Bool = false, i do { selector = try ToolchainSelector(parsing: selectorString) } catch { - return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file is malformed: \(svFile) \(error)"))) + return (nil, .swiftVersionFile(svFile, nil, Error(message: "The swift version file is malformed: \(svFile) \(error)"))) } guard let selector = selector else { - return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file is malformed: \(svFile)"))) + return (nil, .swiftVersionFile(svFile, nil, Error(message: "The swift version file is malformed: \(svFile)"))) } if install { @@ -225,10 +225,10 @@ public func selectToolchain(config: inout Config, globalDefault: Bool = false, i } guard let selectedToolchain = config.listInstalledToolchains(selector: selector).max() else { - return (nil, .swiftVersionFile(svFile, Error(message: "The swift version file didn't select any of the installed toolchains. You can install one with `swiftly install \(selector.description)`."))) + return (nil, .swiftVersionFile(svFile, selector, Error(message: "The swift version file didn't select any of the installed toolchains. You can install one with `swiftly install \(selector.description)`."))) } - return (selectedToolchain, .swiftVersionFile(svFile, nil)) + return (selectedToolchain, .swiftVersionFile(svFile, selector, nil)) } cwd = cwd.deletingLastPathComponent() From 43d620a45388439ca0890679d50c35fe382dccfc Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Thu, 12 Sep 2024 15:50:28 -0400 Subject: [PATCH 18/26] Fix case of empty bin directory when checking for overwrite --- Sources/Swiftly/Init.swift | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Sources/Swiftly/Init.swift b/Sources/Swiftly/Init.swift index 8b89438d..a4b7547f 100644 --- a/Sources/Swiftly/Init.swift +++ b/Sources/Swiftly/Init.swift @@ -56,7 +56,7 @@ internal struct Init: SwiftlyCommand { // Ensure swiftly doesn't overwrite any existing executables without getting confirmation first. let swiftlyBinDir = Swiftly.currentPlatform.swiftlyBinDir - let swiftlyBinDirContents = try FileManager.default.contentsOfDirectory(atPath: swiftlyBinDir.path) + let swiftlyBinDirContents = (try? FileManager.default.contentsOfDirectory(atPath: swiftlyBinDir.path)) ?? [String]() let willBeOverwritten = Set(proxyList + ["swiftly"]).intersection(swiftlyBinDirContents) if !willBeOverwritten.isEmpty && !overwrite { SwiftlyCore.print("The following existing executables will be overwritten:") From 2e3c59dd8385ba00dee69f580e58ce3005c9c71a Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Fri, 20 Sep 2024 10:48:38 -0400 Subject: [PATCH 19/26] Remove +install selector option from swift run in favour of regular `swiftly install`. Guard automatic creation of .swift-version file from `swiftly use` around a prompt overridable using an `--assume-yes`. Minor cleanup --- DESIGN.md | 10 ++--- .../SwiftlyDocs.docc/swiftly-cli-reference.md | 13 +++--- Sources/Swiftly/Install.swift | 2 +- Sources/Swiftly/Run.swift | 45 +++++-------------- Sources/Swiftly/Uninstall.swift | 2 +- Sources/Swiftly/Use.swift | 38 +++++++--------- Sources/SwiftlyCore/Platform.swift | 2 +- Tests/SwiftlyTests/RunTests.swift | 24 ++++------ 8 files changed, 51 insertions(+), 85 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index e6a814c9..97921380 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -326,14 +326,14 @@ swiftly run swift build +5.10.1 # Runs swift build with the 5.10.1 toolchain A few notes about the '+' prefix. First, if a literal '+' prefix should be sent directly to the tool as an argument then it is escaped by doubling it with '++'. An argument with only '++' is ignored entirely, and any additional arguments are sent directly to the command without any further inspection of their prefixes. This is analogous to the special '--' token that certain argument parsers accept so that they don't interpret anything following that token as command flags or options. -If the selected toolchain is not installed then swiftly will exit with a message indicating that you need to run `swiftly install x.y.z` to install it. However, if you enter a special `+install` token then swiftly will automatically download and install the toolchain if it isn't already present. +If the selected toolchain is not installed then swiftly will exit with a message indicating that you need to run `swiftly install x.y.z` to install it. ``` -# Download and install the latest main snapshot toolchain and run 'swift build' to build the package with it. -swiftly run swift build +main-snapshot +install +# Use the latest main snapshot toolchain and run 'swift build' to build the package with it. +swiftly run swift build +main-snapshot -# Generate makefiles with the latest released Swift toolchain, download and install it if necessary -swiftly run +latest +install cmake -G "Unix Makefile" +# Generate makefiles with the latest released Swift toolchain +swiftly run +latest cmake -G "Unix Makefile" swiftly run +latest make ``` diff --git a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md index 887435e1..7db3879d 100644 --- a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md +++ b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md @@ -143,10 +143,10 @@ Note that listing available snapshots before 6.0 is unsupported. ## use -Set the active toolchain. If no toolchain is provided, print the currently in-use toolchain, if any. +Set the in-use toolchain. If no toolchain is provided, print the currently in-use toolchain, if any. ``` -swiftly use [--print-location] [--global-default] [] [--version] [--help] +swiftly use [--print-location] [--global-default] [--assume-yes] [] [--version] [--help] ``` **--print-location:** @@ -159,6 +159,11 @@ swiftly use [--print-location] [--global-default] [] [--version] [--h *Use the global default, ignoring any .swift-version files.* +**--assume-yes:** + +*Disable confirmation prompts by assuming 'yes'* + + **toolchain:** *The toolchain to use.* @@ -469,9 +474,7 @@ You can also override the selection mechanisms temporarily for the duration of t $ swiftly run swift build +latest $ swiftly run swift build +5.10.1 -The first command builds the swift package with the latest toolchain and the second selects the 5.10.1 toolchain. Note that if these aren't installed then run will fail with an error message. You can pre-install the toolchain using `swiftly install ` to ensure success. There is also a `+install` argument that will automatically download and install the toolchain if necessary. - - $ swiftly run swift build +latest +install +The first command builds the swift package with the latest toolchain and the second selects the 5.10.1 toolchain. Note that if these aren't installed then run will fail with an error message. You can pre-install the toolchain using `swiftly install ` to ensure success. If the command that you are running needs the arguments with the '+' prefixes then you can escape it by doubling the '++'. diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index b9aa1fba..cfbb1a87 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -242,7 +242,7 @@ struct Install: SwiftlyCommand { // --use argument was provided. if useInstalledToolchain || config.inUse == nil { // TODO: consider adding the global default option to this commands flags - try await Use.execute(version, false, &config) + try await Use.execute(version, globalDefault: false, &config) } SwiftlyCore.print("\(version) installed successfully!") diff --git a/Sources/Swiftly/Run.swift b/Sources/Swiftly/Run.swift index f165b39b..4f4f67fe 100644 --- a/Sources/Swiftly/Run.swift +++ b/Sources/Swiftly/Run.swift @@ -35,10 +35,7 @@ internal struct Run: SwiftlyCommand { The first command builds the swift package with the latest toolchain and the second selects the \ 5.10.1 toolchain. Note that if these aren't installed then run will fail with an error message. \ - You can pre-install the toolchain using `swiftly install ` to ensure success. There is \ - also a `+install` argument that will automatically download and install the toolchain if necessary. - - $ swiftly run swift build +latest +install + You can pre-install the toolchain using `swiftly install ` to ensure success. If the command that you are running needs the arguments with the '+' prefixes then you can escape \ it by doubling the '++'. @@ -62,36 +59,19 @@ internal struct Run: SwiftlyCommand { var config = try Config.load() - let (command, selector, install) = try extractProxyArguments(command: self.command) + let (command, selector) = try extractProxyArguments(command: self.command) let toolchain: ToolchainVersion? if let selector = selector { - if install { - let version = try await Install.resolve(config: config, selector: selector) - let postInstallScript = try await Install.execute(version: version, &config, useInstalledToolchain: false, verifySignature: true) - if let postInstallScript = postInstallScript { - throw Error(message: """ - - There are some system dependencies that should be installed before using this toolchain. - You can run the following script as the system administrator (e.g. root) to prepare - your system: - - \(postInstallScript) - """) - } - - toolchain = version - } else { - let matchedToolchain = config.listInstalledToolchains(selector: selector).max() - guard let matchedToolchain = matchedToolchain else { - throw Error(message: "The selected toolchain \(selector.description) didn't match any of the installed toolchains. You can install it by adding '+install' to your command, or `swiftly install \(selector.description)`") - } - - toolchain = matchedToolchain + let matchedToolchain = config.listInstalledToolchains(selector: selector).max() + guard let matchedToolchain = matchedToolchain else { + throw Error(message: "The selected toolchain \(selector.description) didn't match any of the installed toolchains. You can install it with `swiftly install \(selector.description)`") } + + toolchain = matchedToolchain } else { - let (version, result) = try await selectToolchain(config: &config, install: install) + let (version, result) = try await selectToolchain(config: &config) // Abort on any errors relating to swift version files if case let .swiftVersionFile(_, _, error) = result, let error = error { @@ -124,8 +104,8 @@ internal struct Run: SwiftlyCommand { } } -public func extractProxyArguments(command: [String]) throws -> (command: [String], selector: ToolchainSelector?, install: Bool) { - var args: (command: [String], selector: ToolchainSelector?, install: Bool) = (command: [], nil, false) +public func extractProxyArguments(command: [String]) throws -> (command: [String], selector: ToolchainSelector?) { + var args: (command: [String], selector: ToolchainSelector?) = (command: [], nil) var disableEscaping = false @@ -140,11 +120,6 @@ public func extractProxyArguments(command: [String]) throws -> (command: [String continue } - if !disableEscaping && c == "+install" { - args.install = true - continue - } - if !disableEscaping && c.hasPrefix("+") { args.selector = try ToolchainSelector(parsing: String(c.dropFirst())) continue diff --git a/Sources/Swiftly/Uninstall.swift b/Sources/Swiftly/Uninstall.swift index 1bfb8ff2..b330d7bb 100644 --- a/Sources/Swiftly/Uninstall.swift +++ b/Sources/Swiftly/Uninstall.swift @@ -100,7 +100,7 @@ struct Uninstall: SwiftlyCommand { ?? config.listInstalledToolchains(selector: .latest).filter({ !toolchains.contains($0) }).max() ?? config.installedToolchains.filter({ !toolchains.contains($0) }).max() { - try await Use.execute(toUse, true, &config) + try await Use.execute(toUse, globalDefault: true, &config) } else { // If there are no more toolchains installed, just unuse the currently active toolchain. config.inUse = nil diff --git a/Sources/Swiftly/Use.swift b/Sources/Swiftly/Use.swift index d77aea57..95e6663a 100644 --- a/Sources/Swiftly/Use.swift +++ b/Sources/Swiftly/Use.swift @@ -4,7 +4,7 @@ import SwiftlyCore internal struct Use: SwiftlyCommand { public static var configuration = CommandConfiguration( - abstract: "Set the active toolchain. If no toolchain is provided, print the currently in-use toolchain, if any." + abstract: "Set the in-use toolchain. If no toolchain is provided, print the currently in-use toolchain, if any." ) @Flag(name: .shortAndLong, help: "Print the location of the in-use toolchain. This is valid only when there is no toolchain argument.") @@ -13,6 +13,8 @@ internal struct Use: SwiftlyCommand { @Flag(name: .shortAndLong, help: "Use the global default, ignoring any .swift-version files.") var globalDefault: Bool = false + @OptionGroup var root: GlobalOptions + @Argument(help: ArgumentHelp( "The toolchain to use.", discussion: """ @@ -98,11 +100,11 @@ internal struct Use: SwiftlyCommand { return } - try await Self.execute(toolchain, self.globalDefault, &config) + try await Self.execute(toolchain, globalDefault: self.globalDefault, assumeYes: self.root.assumeYes, &config) } /// Use a toolchain. This method can modify and save the input config. - internal static func execute(_ toolchain: ToolchainVersion, _ globalDefault: Bool, _ config: inout Config) async throws { + internal static func execute(_ toolchain: ToolchainVersion, globalDefault: Bool, assumeYes: Bool = true, _ config: inout Config) async throws { let (selectedVersion, result) = try await selectToolchain(config: &config, globalDefault: globalDefault) if let selectedVersion = selectedVersion { @@ -116,13 +118,22 @@ internal struct Use: SwiftlyCommand { // We don't care in this case if there were any problems with the swift version files, just overwrite it with the new value try toolchain.name.write(to: versionFile, atomically: true, encoding: .utf8) } else if let newVersionFile = findNewVersionFile(), !globalDefault { + if !assumeYes { + SwiftlyCore.print("A new file `\(newVersionFile)` will be created to set the new in-use toolchain for this project. Alternatively, you can set your default globally with the `--global-default` flag. Proceed with creating this file?") + + guard SwiftlyCore.promptForConfirmation(defaultBehavior: true) else { + SwiftlyCore.print("Aborting setting in-use toolchain") + return + } + } + try toolchain.name.write(to: newVersionFile, atomically: true, encoding: .utf8) } else { config.inUse = toolchain try config.save() } - var message = "Set the used toolchain to \(toolchain)" + var message = "Set the in-use toolchain to \(toolchain)" if let selectedVersion = selectedVersion { message += " (was \(selectedVersion.name))" } @@ -175,7 +186,7 @@ public enum ToolchainSelectionResult { /// If such a case happens then the toolchain version in the tuple will be nil, but the /// result will be .swiftVersionFile and a detailed error about the problem. This error /// can be thrown by the client, or ignored. -public func selectToolchain(config: inout Config, globalDefault: Bool = false, install: Bool = false) async throws -> (ToolchainVersion?, ToolchainSelectionResult) { +public func selectToolchain(config: inout Config, globalDefault: Bool = false) async throws -> (ToolchainVersion?, ToolchainSelectionResult) { if !globalDefault { var cwd = URL(fileURLWithPath: FileManager.default.currentDirectoryPath) @@ -209,23 +220,8 @@ public func selectToolchain(config: inout Config, globalDefault: Bool = false, i return (nil, .swiftVersionFile(svFile, nil, Error(message: "The swift version file is malformed: \(svFile)"))) } - if install { - let version = try await Install.resolve(config: config, selector: selector) - let postInstallScript = try await Install.execute(version: version, &config, useInstalledToolchain: false, verifySignature: true) - if let postInstallScript = postInstallScript { - throw Error(message: """ - - There are some system dependencies that should be installed before using this toolchain. - You can run the following script as the system administrator (e.g. root) to prepare - your system: - - \(postInstallScript) - """) - } - } - guard let selectedToolchain = config.listInstalledToolchains(selector: selector).max() else { - return (nil, .swiftVersionFile(svFile, selector, Error(message: "The swift version file didn't select any of the installed toolchains. You can install one with `swiftly install \(selector.description)`."))) + return (nil, .swiftVersionFile(svFile, selector, Error(message: "The swift version file `\(svFile.path)` uses toolchain version \(selector), but it doesn't match any of the installed toolchains. You can install the toolchain with `swiftly install`."))) } return (selectedToolchain, .swiftVersionFile(svFile, selector, nil)) diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index 9042a8c4..d0f7a39f 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -142,7 +142,7 @@ extension Platform { // The toolchain goes to the beginning of the PATH var newPath = newEnv["PATH"] ?? "" if !newPath.hasPrefix(tcPath.path + ":") { - newPath = ([tcPath.path] + newPath.split(separator: ":").map { String($0) }).joined(separator: ":") + newPath = "\(tcPath.path):\(newPath)" } newEnv["PATH"] = newPath diff --git a/Tests/SwiftlyTests/RunTests.swift b/Tests/SwiftlyTests/RunTests.swift index 7b92c38b..846509c1 100644 --- a/Tests/SwiftlyTests/RunTests.swift +++ b/Tests/SwiftlyTests/RunTests.swift @@ -58,34 +58,28 @@ final class RunTests: SwiftlyTests { /// Tests the extraction of proxy arguments from the run command arguments. func testExtractProxyArguments() throws { - var (command, selector, install) = try extractProxyArguments(command: ["swift", "build"]) + var (command, selector) = try extractProxyArguments(command: ["swift", "build"]) XCTAssertEqual(["swift", "build"], command) - XCTAssertEqual(false, install) XCTAssertEqual(nil, selector) - (command, selector, install) = try extractProxyArguments(command: ["swift", "+1.2.3", "build"]) + (command, selector) = try extractProxyArguments(command: ["swift", "+1.2.3", "build"]) XCTAssertEqual(["swift", "build"], command) - XCTAssertEqual(false, install) XCTAssertEqual(try! ToolchainSelector(parsing: "1.2.3"), selector) - (command, selector, install) = try extractProxyArguments(command: ["swift", "build", "+latest"]) + (command, selector) = try extractProxyArguments(command: ["swift", "build", "+latest"]) XCTAssertEqual(["swift", "build"], command) - XCTAssertEqual(false, install) XCTAssertEqual(try! ToolchainSelector(parsing: "latest"), selector) - (command, selector, install) = try extractProxyArguments(command: ["+5.6", "swift", "build"]) + (command, selector) = try extractProxyArguments(command: ["+5.6", "swift", "build"]) XCTAssertEqual(["swift", "build"], command) - XCTAssertEqual(false, install) XCTAssertEqual(try! ToolchainSelector(parsing: "5.6"), selector) - (command, selector, install) = try extractProxyArguments(command: ["swift", "++1.2.3", "build"]) + (command, selector) = try extractProxyArguments(command: ["swift", "++1.2.3", "build"]) XCTAssertEqual(["swift", "+1.2.3", "build"], command) - XCTAssertEqual(false, install) XCTAssertEqual(nil, selector) - (command, selector, install) = try extractProxyArguments(command: ["swift", "++", "+1.2.3", "build"]) + (command, selector) = try extractProxyArguments(command: ["swift", "++", "+1.2.3", "build"]) XCTAssertEqual(["swift", "+1.2.3", "build"], command) - XCTAssertEqual(false, install) XCTAssertEqual(nil, selector) do { @@ -98,14 +92,12 @@ final class RunTests: SwiftlyTests { XCTAssert(false) } catch {} - (command, selector, install) = try extractProxyArguments(command: ["swift", "+1.2.3", "+install", "build"]) + (command, selector) = try extractProxyArguments(command: ["swift", "+1.2.3", "build"]) XCTAssertEqual(["swift", "build"], command) - XCTAssertEqual(true, install) XCTAssertEqual(try! ToolchainSelector(parsing: "1.2.3"), selector) - (command, selector, install) = try extractProxyArguments(command: ["swift", "+install", "build"]) + (command, selector) = try extractProxyArguments(command: ["swift", "build"]) XCTAssertEqual(["swift", "build"], command) - XCTAssertEqual(true, install) XCTAssertEqual(nil, selector) } } From 9dd640c5c8835de04b1085cbd46f172453748e67 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Tue, 17 Sep 2024 14:26:53 -0400 Subject: [PATCH 20/26] Import GPG keys on every install to get new signing keys from swift.org Fix all of the swift.org urls so that they use www.swift.org to avoid redirection --- Sources/LinuxPlatform/Linux.swift | 29 +++--------------------- Tests/SwiftlyTests/HTTPClientTests.swift | 4 ++-- 2 files changed, 5 insertions(+), 28 deletions(-) diff --git a/Sources/LinuxPlatform/Linux.swift b/Sources/LinuxPlatform/Linux.swift index 072c2a7e..e20eec93 100644 --- a/Sources/LinuxPlatform/Linux.swift +++ b/Sources/LinuxPlatform/Linux.swift @@ -193,44 +193,21 @@ public struct Linux: Platform { throw Error(message: msg) } - let foundKeys = (try? self.runProgram( - "gpg", - "--list-keys", - "swift-infrastructure@forums.swift.org", - "swift-infrastructure@swift.org", - quiet: true - )) != nil - if !foundKeys { - // Import the swift keys if they aren't here already + // Import the latest swift keys, but only once per session, which will help with the performance in tests + if !swiftGPGKeysRefreshed { let tmpFile = self.getTempFilePath() FileManager.default.createFile(atPath: tmpFile.path, contents: nil, attributes: [.posixPermissions: 0o600]) defer { try? FileManager.default.removeItem(at: tmpFile) } - guard let url = URL(string: "https://swift.org/keys/all-keys.asc") else { + guard let url = URL(string: "https://www.swift.org/keys/all-keys.asc") else { throw Error(message: "malformed URL to the swift gpg keys") } try await httpClient.downloadFile(url: url, to: tmpFile) try self.runProgram("gpg", "--import", tmpFile.path, quiet: true) - } - // We only need to refresh the keys once per session, which will help with performance in tests - if !swiftGPGKeysRefreshed { - SwiftlyCore.print("Refreshing Swift PGP keys...") - do { - try self.runProgram( - "gpg", - "--quiet", - "--keyserver", - "hkp://keyserver.ubuntu.com", - "--refresh-keys", - "Swift" - ) - } catch { - throw Error(message: "Failed to refresh PGP keys: \(error)") - } swiftGPGKeysRefreshed = true } } diff --git a/Tests/SwiftlyTests/HTTPClientTests.swift b/Tests/SwiftlyTests/HTTPClientTests.swift index e72155ae..1d666d68 100644 --- a/Tests/SwiftlyTests/HTTPClientTests.swift +++ b/Tests/SwiftlyTests/HTTPClientTests.swift @@ -7,7 +7,7 @@ final class HTTPClientTests: SwiftlyTests { // GIVEN: we have a swiftly http client // WHEN: we make get request for a particular type of JSON var releases: [SwiftOrgRelease] = try await SwiftlyCore.httpClient.getFromJSON( - url: "https://swift.org/api/v1/install/releases.json", + url: "https://www.swift.org/api/v1/install/releases.json", type: [SwiftOrgRelease].self, headers: [:] ) @@ -19,7 +19,7 @@ final class HTTPClientTests: SwiftlyTests { var exceptionThrown = false do { releases = try await SwiftlyCore.httpClient.getFromJSON( - url: "https://swift.org/api/v1/install/releases-invalid.json", + url: "https://www.swift.org/api/v1/install/releases-invalid.json", type: [SwiftOrgRelease].self, headers: [:] ) From 5e615ea70634166e7abd7919475821c6dd4be0eb Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Fri, 18 Oct 2024 08:23:30 -0400 Subject: [PATCH 21/26] Make recommended documentation changes. Fix symlink target selection for swiftly when it is system managed --- DESIGN.md | 6 +++--- Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md | 4 ++-- Sources/Swiftly/Init.swift | 8 ++++---- Sources/Swiftly/Install.swift | 2 +- Sources/Swiftly/Run.swift | 5 ++--- Sources/SwiftlyCore/Platform.swift | 6 +++--- 6 files changed, 15 insertions(+), 16 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index 97921380..1c76c4d4 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -147,11 +147,11 @@ Installing a specific snapshot from a swift version development branch ##### Installing the version from the `.swift-version` file -A package could have a swift version file that specifies the recommended toolchain version. A swiftly install with no version will search for a version file and install that version. +A package could have a ".swift-version" file that specifies the recommended toolchain version. A swiftly install with no version will search for a version file and install that version. `swiftly install` -If no swift version file can be found then the installation fails indicating that it couldn't fine the file. +If no ".swift-version" file can be found then the installation fails indicating that it couldn't fine the file. #### uninstall @@ -223,7 +223,7 @@ To use the latest installed main snapshot, leave off the date: `swiftly use main-snapshot` -The use subcommand also supports `.swift-version` files. If version file is present in the current working directory, or an ancestory directory, then swiftly will update that file with the new version to use. This can be a useful feature for a team to share and align on toolchain versions with git. As a special case, if swiftly could not find a version file, but it could find a Package.swift file it will create a new version file for you in the package and set that to the requested toolchain version. +The use subcommand also supports `.swift-version` files. If a ".swift-version" file is present in the current working directory, or an ancestory directory, then swiftly will update that file with the new version to use. This can be a useful feature for a team to share and align on toolchain versions with git. As a special case, if swiftly could not find a version file, but it could find a Package.swift file it will create a new version file for you in the package and set that to the requested toolchain version. Note: The `.swift-version` file mechanisms can be overridden using the `--global-default` flag so that your swiftly installation's default toolchain can be set explicitly. diff --git a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md index 7db3879d..dbe97f63 100644 --- a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md +++ b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md @@ -53,7 +53,7 @@ Likewise, the latest snapshot associated with a given development branch can be $ swiftly install 5.7-snapshot $ swiftly install main-snapshot - Install whatever toolchain is current selected, such as a .swift-version file: + Install whatever toolchain is currently selected, such as the the one in the .swift-version file: $ swiftly install @@ -456,7 +456,7 @@ swiftly run ... [--version] [--help] *Run a command while proxying to the selected toolchain commands.* -Run a command with a selected toolchain, so that all toolchain commands are become the default added to the system path and other common environment variables. +Run a command with a selected toolchain. The toolchain commands become the default in the system path. You can run one of the usual toolchain commands directly: diff --git a/Sources/Swiftly/Init.swift b/Sources/Swiftly/Init.swift index a4b7547f..9a354022 100644 --- a/Sources/Swiftly/Init.swift +++ b/Sources/Swiftly/Init.swift @@ -124,10 +124,10 @@ internal struct Init: SwiftlyCommand { let swiftlyBin = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swiftly", isDirectory: false) let cmd = URL(fileURLWithPath: CommandLine.arguments[0]) - let systemManaged = try Swiftly.currentPlatform.isSystemManagedBinary(cmd.path) + let systemManagedSwiftlyBin = try Swiftly.currentPlatform.systemManagedBinary(CommandLine.arguments[0]) // Don't move the binary if it's already in the right place, this is being invoked inside an xctest, or it is a system managed binary - if cmd != swiftlyBin && !cmd.path.hasSuffix("xctest") && !systemManaged { + if cmd != swiftlyBin && !cmd.path.hasSuffix("xctest") && systemManagedSwiftlyBin == nil { SwiftlyCore.print("Moving swiftly into the installation directory...") if swiftlyBin.fileExists() { @@ -146,8 +146,8 @@ internal struct Init: SwiftlyCommand { if !cmd.path.hasSuffix("xctest") { SwiftlyCore.print("Setting up toolchain proxies...") - let proxyTo = if systemManaged { - cmd.path + let proxyTo = if let systemManagedSwiftlyBin = systemManagedSwiftlyBin { + systemManagedSwiftlyBin } else { swiftlyBin.path } diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index cfbb1a87..2e565aa1 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -40,7 +40,7 @@ struct Install: SwiftlyCommand { $ swiftly install 5.7-snapshot $ swiftly install main-snapshot - Install whatever toolchain is current selected, such as a .swift-version file: + Install whatever toolchain is currently selected, such as the the one in the .swift-version file: $ swiftly install """ diff --git a/Sources/Swiftly/Run.swift b/Sources/Swiftly/Run.swift index 4f4f67fe..c60c5da1 100644 --- a/Sources/Swiftly/Run.swift +++ b/Sources/Swiftly/Run.swift @@ -11,9 +11,8 @@ internal struct Run: SwiftlyCommand { "Run a command while proxying to the selected toolchain commands.", discussion: """ - Run a command with a selected toolchain, so that all toolchain commands \ - are become the default added to the system path and other common environment \ - variables. + Run a command with a selected toolchain. The toolchain commands \ + become the default in the system path. You can run one of the usual toolchain commands directly: diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index d0f7a39f..8e995387 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -261,7 +261,7 @@ extension Platform { } } - public func isSystemManagedBinary(_ cmd: String) throws -> Bool { + public func systemManagedBinary(_ cmd: String) throws -> String? { let userHome = FileManager.default.homeDirectoryForCurrentUser let binLocs = [cmd] + ProcessInfo.processInfo.environment["PATH"]!.components(separatedBy: ":").map { $0 + "/" + cmd } var bin: String? @@ -278,10 +278,10 @@ extension Platform { // If the binary is in the user's home directory, or is not in system locations ("/usr", "/opt", "/bin") // then it is expected to be outside of a system package location and we manage the binary ourselves. if bin.hasPrefix(userHome.path + "/") || (!bin.hasPrefix("/usr") && !bin.hasPrefix("/opt") && !bin.hasPrefix("/bin")) { - return false + return nil } - return true + return bin } #endif From 6d6050e3e0335a7d0e5250053a0280e9fbe2ab8b Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sat, 19 Oct 2024 07:58:41 -0400 Subject: [PATCH 22/26] Provide a better error message on swiftly install with no version Print the error in a better way --- Sources/Swiftly/Install.swift | 2 +- Sources/Swiftly/Proxy.swift | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index 2e565aa1..3d14f8ce 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -96,7 +96,7 @@ struct Install: SwiftlyCommand { throw Error(message: "Internal error selecting toolchain to install.") } } else { - throw Error(message: "There is no toolchain selected from a .swift-version file to install.") + throw Error(message: "Swiftly couldn't determine the toolchain version to install. Please set a version like this and try again: `swiftly install latest`") } } diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift index 3289a8ab..b91bdd4f 100644 --- a/Sources/Swiftly/Proxy.swift +++ b/Sources/Swiftly/Proxy.swift @@ -51,6 +51,9 @@ public enum Proxy { try await Swiftly.currentPlatform.proxy(toolchain, binName, Array(CommandLine.arguments[1...])) } catch let terminated as RunProgramError { exit(terminated.exitCode) + } catch let error as Error { + SwiftlyCore.print(error.message) + exit(1) } catch { SwiftlyCore.print("\(error)") exit(1) From 69897966f9c0aa0ddaed9b77590dc28bee8a9383 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Sun, 20 Oct 2024 11:06:17 -0400 Subject: [PATCH 23/26] Update README, and add documentation for the new run subcommand --- Documentation/SwiftlyDocs.docc/SwiftlyDocs.md | 1 + .../SwiftlyDocs.docc/getting-started.md | 23 +++++- .../SwiftlyDocs.docc/use-toolchains.md | 79 +++++++++++++++++++ README.md | 41 ++++------ 4 files changed, 114 insertions(+), 30 deletions(-) create mode 100644 Documentation/SwiftlyDocs.docc/use-toolchains.md diff --git a/Documentation/SwiftlyDocs.docc/SwiftlyDocs.md b/Documentation/SwiftlyDocs.docc/SwiftlyDocs.md index 8af36940..bd0b4da0 100644 --- a/Documentation/SwiftlyDocs.docc/SwiftlyDocs.md +++ b/Documentation/SwiftlyDocs.docc/SwiftlyDocs.md @@ -13,6 +13,7 @@ Install and manage your Swift programming language toolchains. ### HOWTOS - +- - - - diff --git a/Documentation/SwiftlyDocs.docc/getting-started.md b/Documentation/SwiftlyDocs.docc/getting-started.md index 7d48eab1..b66d50e3 100644 --- a/Documentation/SwiftlyDocs.docc/getting-started.md +++ b/Documentation/SwiftlyDocs.docc/getting-started.md @@ -27,9 +27,11 @@ $ swift --version Swift version 5.8.1 (swift-5.8.1-RELEASE) Target: x86_64-unknown-linux-gnu + +$ swift build # Build with the latest (5.8.1) toolchain ``` -Or, you can install (and use) a swift release: +You can install (and use) another release toolchain: ``` $ swiftly install --use 5.7 @@ -38,12 +40,27 @@ $ swift --version Swift version 5.7.2 (swift-5.7.2-RELEASE) Target: x86_64-unknown-linux-gnu + +$ swift build # Build with the 5.7.2 toolchain ``` -There's also an option to install the latest snapshot release and get access to the latest features: +Quickly test your package with the latest nightly snapshot to prepare for the next release: ``` $ swiftly install main-snapshot +$ swiftly run swift test +main-snapshot # Run "swift test" with the main-snapshot toolchain +$ swift build # Continue to build with my usual toolchain ``` -> Note: This last example just installed the toolchain. You can run "swiftly use" to switch to it and other installed toolchahins when you're ready. +Uninstall this toolchain after you're finished with it: + +``` +$ swiftly uninstall main-snapshot +``` + +# See Also: + +- [Install Toolchains](install-toolchains) +- [Using Toolchains](use-toolchains) +- [Uninstall Toolchains](uninstall-toolchains) +- [Swiftly CLI Reference](swiftly-cli-reference) diff --git a/Documentation/SwiftlyDocs.docc/use-toolchains.md b/Documentation/SwiftlyDocs.docc/use-toolchains.md new file mode 100644 index 00000000..941b9a80 --- /dev/null +++ b/Documentation/SwiftlyDocs.docc/use-toolchains.md @@ -0,0 +1,79 @@ +# Use Swift Toolchains + +swiftly use and swiftly run + +Swiftly toolchains include a variety of compilers, linkers, debuggers, documentation generators, and other useful tools for working with Swift. Using a toolchain activates it so that when you run toolchain commands they are run with that version. + +When you install a toolchain you can start using it right away. If you don't have any other toolchains installed then it becomes the default. + +``` +$ swiftly install latest +$ swift --version +Swift version 6.0.1 (swift-6.0.1-RELEASE) +Target: aarch64-unknown-linux-gnu +$ swift build # Build with the current toolchain +``` + +When you have more than one toolchain installed then you can choose to use one of them with `swiftly use` for all subsequent commands like this: + +``` +$ swiftly install 5.10.1 +$ swiftly install main-snapshot +$ swiftly use 5.10.1 +$ swift build # Builds with the 5.10.1 toolchain +$ swift test # Tests with the 5.10.1 toolchain +$ swiftly use main-snapshot +$ swift build # Builds with the latest snapshot toolchain on the main branch +$ lldb # Run the debugger from the latest snapshot toolchain +``` + +If you're not certain which toolchain is in-use then use the bare `swiftly use` command to provide details: + +``` +$ swiftly use +Swift 6.0.1 (default) +``` + +You can print the exact toolchain location with the `--print-location` flag: + +``` +$ swiftly use --print-location +/Users/someuser/Library/Developer/Toolchains/swift-5.10.1-RELEASE.xctoolchain +``` + +## Sharing recommended toolchain versions + +Swiftly can create and update a special `.swift-version` file at the top of your git repository so that you can share your toolchain preference with the rest of your team: + +``` +$ cd path/to/git/repository +$ swiftly use 6.0.1 +A new file `path/to/git/repository/.swift-version` will be created to set the new in-use toolchain for this project. +Alternatively, you can set your default globally with the `--global-default` flag. Proceed with creating this file? (Y/n) Y +$ cat .swift-version +6.0.1 +``` + +When a team member uses swiftly with this git repository it can use the correct toolchain version automatically: + +``` +$ cd path/to/git/repository +$ swift --version +Swift version 6.0.1 (swift-6.0.1-RELEASE) +Target: aarch64-unknown-linux-gnu +``` + +If that team member doesn't have the toolchain installed on their system there will be a warning. They can install the selected toolchain automatically like this: + +``` +$ cd path/to/git/repository +$ swiftly install # Installs the version of the toolchain in the .swift-version file +``` + +If you want to temporarily use a toolchain version for one command you can try `swiftly run`. This will build your package with the latest snapshot toolchain: + +``` +$ swiftly run swift build +main-snapshot +``` + +> Note: The toolchain must be installed on your system before you can run with it. diff --git a/README.md b/README.md index 72edab9e..fbc9afa4 100644 --- a/README.md +++ b/README.md @@ -134,50 +134,37 @@ $ swiftly list ### Selecting a toolchain for use -“Using” a toolchain sets it as the active toolchain, meaning it will be the one found via $PATH and invoked via `swift` commands executed in the shell. +“Using” a toolchain sets it as the active toolchain, meaning it will be the one found via $PATH and invoked via `swift` commands executed in the shell. The toolchain must be installed before you can use it. -To use the toolchain associated with the most up-to-date Swift version, the “latest” version can be specified: +You can provide the same version selectors as you used with `swiftly install` to use a toolchain, including exact releacs versions "major.minor.patch", and snapshots. ``` $ swiftly use latest -``` - -To use a specific stable version of Swift already installed, specify the major/minor/patch version: - -``` $ swiftly use 5.3.1 -``` - -To use the latest installed patch version associated with a given major/minor version pair, the patch can be omitted: - -``` $ swiftly use 5.3 -``` - -To use a specific snapshot version, specify the full snapshot version name: - -``` -$ swiftly use 5.3-snapshot-YYYY-MM-DD -``` - -To use the latest installed snapshot associated with a given version, the date can be omitted: - -``` $ swiftly use 5.3-snapshot +$ swiftly use 5.3-snapshot-2022-08-16 +$ swiftly use main-snapshot +$ swiftly use main-snapshot-2024-06-18 ``` -To use a specific main snapshot, specify the full snapshot version name: +After you use a toolchain your commands at the shell will run with that toolchain: ``` -$ swiftly use main-snapshot-YYYY-MM-DD +$ swiftly use x.y.z +$ swift build # Build my package with toolchain version x.y.z +$ clang -c foo.c -o foo.o # Compile this C file using the clang compiler in toolchain version x.y.z +$ lldb # Open the debugger from toolchain version x.y.z ``` -To use the latest installed main snapshot, leave off the date: +If you want to run just one command with a particular toolchain without having to switch back to the one you used previously you can use the `swiftly run` command with the version. This command builds your current package with the latest snapshot toolchain of the current release: ``` -$ swiftly use main-snapshot +$ swiftly run swift build +main-snapshot ``` +The parameter with the "+" indicates that this is the version selector of the toolchain to use and supports the full range of selectors shown above and with the `swiftly install` command. The toolchain must be installed to run a command with that toolchain. + ### Updating a toolchain Update replaces a given toolchain with a later version of that toolchain. For a stable release, this means updating to a later patch, minor, or major version. For snapshots, this means updating to the most recently available snapshot. From 52d081f6767516ec40806b377eebd49c2e3e5887 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Wed, 23 Oct 2024 11:20:12 -0400 Subject: [PATCH 24/26] Prompt before updating the `.swift-version` file. --- Sources/Swiftly/Use.swift | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/Sources/Swiftly/Use.swift b/Sources/Swiftly/Use.swift index 95e6663a..aeecbe5b 100644 --- a/Sources/Swiftly/Use.swift +++ b/Sources/Swiftly/Use.swift @@ -103,18 +103,20 @@ internal struct Use: SwiftlyCommand { try await Self.execute(toolchain, globalDefault: self.globalDefault, assumeYes: self.root.assumeYes, &config) } - /// Use a toolchain. This method can modify and save the input config. + /// Use a toolchain. This method can modify and save the input config and also create/modify a `.swift-version` file. internal static func execute(_ toolchain: ToolchainVersion, globalDefault: Bool, assumeYes: Bool = true, _ config: inout Config) async throws { let (selectedVersion, result) = try await selectToolchain(config: &config, globalDefault: globalDefault) - if let selectedVersion = selectedVersion { - guard selectedVersion != toolchain else { - SwiftlyCore.print("\(toolchain) is already in use") - return + if case let .swiftVersionFile(versionFile, _, _) = result { + if !assumeYes { + SwiftlyCore.print("The file `\(versionFile)` will be updated to set the new in-use toolchain for this project. Alternatively, you can set your default globally with the `--global-default` flag. Proceed with modifying this file?") + + guard SwiftlyCore.promptForConfirmation(defaultBehavior: true) else { + SwiftlyCore.print("Aborting setting in-use toolchain") + return + } } - } - if case let .swiftVersionFile(versionFile, _, _) = result { // We don't care in this case if there were any problems with the swift version files, just overwrite it with the new value try toolchain.name.write(to: versionFile, atomically: true, encoding: .utf8) } else if let newVersionFile = findNewVersionFile(), !globalDefault { From cb0e9237a099f06721f4142639a576761a70b616 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Tue, 5 Nov 2024 18:18:52 -0500 Subject: [PATCH 25/26] Create proxies on toolchain installation, creating only the necessary ones, giving a message about the shell Set only the PATH environment in swiftly run leaving CC and CXX to the user --- DESIGN.md | 20 +++-- .../SwiftlyDocs.docc/swiftly-cli-reference.md | 9 +- Sources/LinuxPlatform/Linux.swift | 6 +- Sources/Swiftly/Init.swift | 38 +------- Sources/Swiftly/Install.swift | 90 +++++++++++++++++-- Sources/Swiftly/Proxy.swift | 21 +---- Sources/Swiftly/Run.swift | 2 +- Sources/Swiftly/Update.swift | 15 +++- Sources/SwiftlyCore/Platform.swift | 10 ++- Tests/SwiftlyTests/RunTests.swift | 9 +- 10 files changed, 125 insertions(+), 95 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index 1c76c4d4..6bf507e0 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -115,7 +115,7 @@ This will install the latest available stable release of Swift. If the latest ve ##### Installing a specific release version of Swift -To install a specific version of Swift, the user can provide it. +To install a specific version of Swift, the user can provide it. If a patch version isn't specified, it’ll install the latest patch version that matches the minor version provided. If a version is already installed that has the same major and minor version, a message will be printed indicating so and directing the user to `swiftly update a.b` if they wish to check for updates. @@ -306,19 +306,21 @@ This command will provide the full path to the directory where the selected tool #### Run with a selected toolchain -There are cases where you might want to run an arbitrary command using a selected toolchain. An example could be that you want to build something with CMake. +There are cases where you might want to run an arbitrary command using a selected toolchain. An example could be that you want to build something with CMake or Autoconf. ``` # CMake -swiftly run cmake -G ninja +swiftly run cmake -G ninja -D CMAKE_C_COMPILER=clang -D CMAKE_CXX_COMPILER=clang++ swiftly run ninja build # Autoconf -swiftly run ./configure -swiftly run make +CC=clang swiftly run ./configure +CC=clang swiftly run make ``` -Swiftly adjusts certain environment variables, such as prefixing the PATH to the selected toolchain directory, and setting the CC and CXX variables to the locations of clang and clang++ in those toolchains so that the build tools use them. If you want to explicitly specify a toolchain for the command you can do that with a selector notation like this: +Swiftly prefixes the PATH to the selected toolchain directory and runs the command so that the toolchain executables are available and have precedence. + +If you want to explicitly specify a toolchain for the command you can do that with a selector notation like this: ``` swiftly run swift build +5.10.1 # Runs swift build with the 5.10.1 toolchain @@ -333,8 +335,8 @@ If the selected toolchain is not installed then swiftly will exit with a message swiftly run swift build +main-snapshot # Generate makefiles with the latest released Swift toolchain -swiftly run +latest cmake -G "Unix Makefile" -swiftly run +latest make +swiftly run +latest cmake -G "Unix Makefile" -D CMAKE_C_COMPILER=clang +CC=clang swiftly run +latest make ``` ## Detailed Design @@ -498,7 +500,7 @@ If the tag is a newer version than the installed one, a prompt indicating the ne $ dpkg --status libcurl4 ``` -If the exit code of the previous command was 0, then we know the dependency exists and can return true. If it wasn't, then we call fall back to attempting to locate the library via `pkg-config`: +If the exit code of the previous command was 0, then we know the dependency exists and can return true. If it wasn't, then we can fall back to attempting to locate the library via `pkg-config`: ``` $ pkg-config --exists libcurl diff --git a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md index d2fb43ca..827a1ecc 100644 --- a/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md +++ b/Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md @@ -23,7 +23,7 @@ swiftly [--version] [--help] Install a new toolchain. ``` -swiftly install [] [--use] [--verify|no-verify] [--post-install-file=] [--version] [--help] +swiftly install [] [--use] [--verify|no-verify] [--post-install-file=] [--assume-yes] [--version] [--help] ``` **version:** @@ -76,6 +76,11 @@ If the toolchain that is installed has extra post installation steps they they w written to this file as commands that can be run after the installation. +**--assume-yes:** + +*Disable confirmation prompts by assuming 'yes'* + + **--version:** *Show the version.* @@ -456,7 +461,7 @@ You can run one of the usual toolchain commands directly: Or you can run another program (or script) that runs one or more toolchain commands: - $ swiftly run make # Builds targets using clang/swiftc + $ CC=clang swiftly run make # Builds targets using clang $ swiftly run ./build-things.sh # Script invokes 'swift build' to create certain product binaries Toolchain selection is determined by swift version files `.swift-version`, with a default global as the fallback. See the `swiftly use` command for more details. diff --git a/Sources/LinuxPlatform/Linux.swift b/Sources/LinuxPlatform/Linux.swift index 4d9345c3..5f1e6a99 100644 --- a/Sources/LinuxPlatform/Linux.swift +++ b/Sources/LinuxPlatform/Linux.swift @@ -14,8 +14,7 @@ public struct Linux: Platform { return URL(fileURLWithPath: dir) } else { return FileManager.default.homeDirectoryForCurrentUser - .appendingPathComponent(".local", isDirectory: true) - .appendingPathComponent("share", isDirectory: true) + .appendingPathComponent(".local/share", isDirectory: true) } } @@ -23,8 +22,7 @@ public struct Linux: Platform { SwiftlyCore.mockedHomeDir.map { $0.appendingPathComponent("bin", isDirectory: true) } ?? ProcessInfo.processInfo.environment["SWIFTLY_BIN_DIR"].map { URL(fileURLWithPath: $0) } ?? FileManager.default.homeDirectoryForCurrentUser - .appendingPathComponent(".local", isDirectory: true) - .appendingPathComponent("bin", isDirectory: true) + .appendingPathComponent(".local/share/swiftly/bin", isDirectory: true) } public var swiftlyToolchainsDir: URL { diff --git a/Sources/Swiftly/Init.swift b/Sources/Swiftly/Init.swift index 9a354022..374984eb 100644 --- a/Sources/Swiftly/Init.swift +++ b/Sources/Swiftly/Init.swift @@ -54,10 +54,9 @@ internal struct Init: SwiftlyCommand { } } - // Ensure swiftly doesn't overwrite any existing executables without getting confirmation first. let swiftlyBinDir = Swiftly.currentPlatform.swiftlyBinDir let swiftlyBinDirContents = (try? FileManager.default.contentsOfDirectory(atPath: swiftlyBinDir.path)) ?? [String]() - let willBeOverwritten = Set(proxyList + ["swiftly"]).intersection(swiftlyBinDirContents) + let willBeOverwritten = Set(["swiftly"]).intersection(swiftlyBinDirContents) if !willBeOverwritten.isEmpty && !overwrite { SwiftlyCore.print("The following existing executables will be overwritten:") @@ -142,30 +141,6 @@ internal struct Init: SwiftlyCommand { } } - // Don't create the proxies in the tests - if !cmd.path.hasSuffix("xctest") { - SwiftlyCore.print("Setting up toolchain proxies...") - - let proxyTo = if let systemManagedSwiftlyBin = systemManagedSwiftlyBin { - systemManagedSwiftlyBin - } else { - swiftlyBin.path - } - - for p in proxyList { - let proxy = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent(p) - - if proxy.fileExists() { - try FileManager.default.removeItem(at: proxy) - } - - try FileManager.default.createSymbolicLink( - atPath: proxy.path, - withDestinationPath: proxyTo - ) - } - } - if overwrite || !FileManager.default.fileExists(atPath: envFile.path) { SwiftlyCore.print("Creating shell environment file for the user...") var env = "" @@ -241,17 +216,6 @@ internal struct Init: SwiftlyCommand { \(sourceLine) """) - -#if os(macOS) - SwiftlyCore.print(""" - NOTE: On macOS it is possible that the shell will pick up the system Swift on the path - instead of the one that swiftly has installed for you. You can run the 'hash -r' - command to update the shell with the latest PATHs. - - hash -r - - """) -#endif } } } diff --git a/Sources/Swiftly/Install.swift b/Sources/Swiftly/Install.swift index 32d6b281..489b1366 100644 --- a/Sources/Swiftly/Install.swift +++ b/Sources/Swiftly/Install.swift @@ -62,8 +62,10 @@ struct Install: SwiftlyCommand { )) var postInstallFile: String? + @OptionGroup var root: GlobalOptions + private enum CodingKeys: String, CodingKey { - case version, use, verify, postInstallFile + case version, use, verify, postInstallFile, root } mutating func run() async throws { @@ -92,14 +94,25 @@ struct Install: SwiftlyCommand { } let toolchainVersion = try await Self.resolve(config: config, selector: selector) - let postInstallScript = try await Self.execute( + let (postInstallScript, pathChanged) = try await Self.execute( version: toolchainVersion, &config, useInstalledToolchain: self.use, - verifySignature: self.verify + verifySignature: self.verify, + assumeYes: self.root.assumeYes ) - if let postInstallScript = postInstallScript { + if pathChanged { + SwiftlyCore.print(""" + NOTE: We have updated some elements in your path and your shell may not yet be + aware of the changes. You can run this command to update your shell. + + hash -r + + """) + } + + if let postInstallScript { guard let postInstallFile = self.postInstallFile else { throw Error(message: """ @@ -119,11 +132,12 @@ struct Install: SwiftlyCommand { version: ToolchainVersion, _ config: inout Config, useInstalledToolchain: Bool, - verifySignature: Bool - ) async throws -> String? { + verifySignature: Bool, + assumeYes: Bool + ) async throws -> (postInstall: String?, pathChanged: Bool) { guard !config.installedToolchains.contains(version) else { SwiftlyCore.print("\(version) is already installed.") - return nil + return (nil, false) } // Ensure the system is set up correctly before downloading it. Problems that prevent installation @@ -224,6 +238,66 @@ struct Install: SwiftlyCommand { try Swiftly.currentPlatform.install(from: tmpFile, version: version) + var pathChanged = false + + // Don't create the proxies in the tests + if CommandLine.arguments.count > 0 && !CommandLine.arguments[0].hasSuffix("xctest") { + // Ensure swiftly doesn't overwrite any existing executables without getting confirmation first. + let swiftlyBin = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent("swiftly", isDirectory: false) + let systemManagedSwiftlyBin = try Swiftly.currentPlatform.systemManagedBinary(CommandLine.arguments[0]) + let swiftlyBinDir = Swiftly.currentPlatform.swiftlyBinDir + let swiftlyBinDirContents = (try? FileManager.default.contentsOfDirectory(atPath: swiftlyBinDir.path)) ?? [String]() + let toolchainBinDir = Swiftly.currentPlatform.findToolchainBinDir(version) + let toolchainBinDirContents = try FileManager.default.contentsOfDirectory(atPath: toolchainBinDir.path) + + let proxyTo = if let systemManagedSwiftlyBin = systemManagedSwiftlyBin { + systemManagedSwiftlyBin + } else { + swiftlyBin.path + } + + let existingProxies = swiftlyBinDirContents.filter { bin in + do { + let linkTarget = try FileManager.default.destinationOfSymbolicLink(atPath: swiftlyBinDir.appendingPathComponent(bin).path) + return linkTarget == proxyTo + } catch { return false } + } + + let overwrite = Set(toolchainBinDirContents).subtracting(existingProxies).intersection(swiftlyBinDirContents) + if !overwrite.isEmpty && !assumeYes { + SwiftlyCore.print("The following existing executables will be overwritten:") + + for executable in overwrite { + SwiftlyCore.print(" \(swiftlyBinDir.appendingPathComponent(executable).path)") + } + + let proceed = SwiftlyCore.readLine(prompt: "Proceed? [y/N]") ?? "n" + + guard proceed == "y" else { + throw Error(message: "Toolchain installation has been cancelled") + } + } + + SwiftlyCore.print("Setting up toolchain proxies...") + + let proxiesToCreate = Set(toolchainBinDirContents).subtracting(swiftlyBinDirContents).union(overwrite) + + for p in proxiesToCreate { + let proxy = Swiftly.currentPlatform.swiftlyBinDir.appendingPathComponent(p) + + if proxy.fileExists() { + try FileManager.default.removeItem(at: proxy) + } + + try FileManager.default.createSymbolicLink( + atPath: proxy.path, + withDestinationPath: proxyTo + ) + + pathChanged = true + } + } + config.installedToolchains.insert(version) try config.save() @@ -236,7 +310,7 @@ struct Install: SwiftlyCommand { } SwiftlyCore.print("\(version) installed successfully!") - return postInstallScript + return (postInstallScript, pathChanged) } /// Utilize the swift.org API along with the provided selector to select a toolchain for install. diff --git a/Sources/Swiftly/Proxy.swift b/Sources/Swiftly/Proxy.swift index b91bdd4f..aca9440d 100644 --- a/Sources/Swiftly/Proxy.swift +++ b/Sources/Swiftly/Proxy.swift @@ -1,25 +1,6 @@ import Foundation import SwiftlyCore -// This is the allowed list of executables that we will proxy -let proxyList = [ - "clang", - "lldb", - "lldb-dap", - "lldb-server", - "clang++", - "sourcekit-lsp", - "clangd", - "swift", - "docc", - "swiftc", - "lld", - "llvm-ar", - "plutil", - "repl_swift", - "wasm-ld", -] - @main public enum Proxy { static func main() async throws { @@ -29,7 +10,7 @@ public enum Proxy { fatalError("Could not determine the binary name for proxying") } - guard proxyList.contains(binName) else { + guard binName != "swiftly" else { // Treat this as a swiftly invocation await Swiftly.main() return diff --git a/Sources/Swiftly/Run.swift b/Sources/Swiftly/Run.swift index c60c5da1..8080930f 100644 --- a/Sources/Swiftly/Run.swift +++ b/Sources/Swiftly/Run.swift @@ -20,7 +20,7 @@ internal struct Run: SwiftlyCommand { Or you can run another program (or script) that runs one or more toolchain commands: - $ swiftly run make # Builds targets using clang/swiftc + $ CC=clang swiftly run make # Builds targets using clang $ swiftly run ./build-things.sh # Script invokes 'swift build' to create certain product binaries Toolchain selection is determined by swift version files `.swift-version`, with a default global \ diff --git a/Sources/Swiftly/Update.swift b/Sources/Swiftly/Update.swift index 3fb4ae27..43e56b55 100644 --- a/Sources/Swiftly/Update.swift +++ b/Sources/Swiftly/Update.swift @@ -108,11 +108,12 @@ struct Update: SwiftlyCommand { } } - let postInstallScript = try await Install.execute( + let (postInstallScript, pathChanged) = try await Install.execute( version: newToolchain, &config, useInstalledToolchain: config.inUse == parameters.oldToolchain, - verifySignature: self.verify + verifySignature: self.verify, + assumeYes: self.root.assumeYes ) try await Uninstall.execute(parameters.oldToolchain, &config) @@ -132,6 +133,16 @@ struct Update: SwiftlyCommand { try Data(postInstallScript.utf8).write(to: URL(fileURLWithPath: postInstallFile), options: .atomic) } + + if pathChanged { + SwiftlyCore.print(""" + NOTE: We have updated some elements in your path and your shell may not yet be + aware of the changes. You can run this command to update your shell. + + hash -r + + """) + } } /// Using the provided toolchain selector and the current config, returns a set of parameters that determines diff --git a/Sources/SwiftlyCore/Platform.swift b/Sources/SwiftlyCore/Platform.swift index 1b7ee8ed..21fce9af 100644 --- a/Sources/SwiftlyCore/Platform.swift +++ b/Sources/SwiftlyCore/Platform.swift @@ -105,6 +105,9 @@ public protocol Platform { /// Find the location where the toolchain should be installed. func findToolchainLocation(_ toolchain: ToolchainVersion) -> URL + + /// Find the location of the toolchain binaries. + func findToolchainBinDir(_ toolchain: ToolchainVersion) -> URL } extension Platform { @@ -150,10 +153,6 @@ extension Platform { } newEnv["PATH"] = newPath - // Add certain common environment variables that can be used to proxy to the toolchain - newEnv["CC"] = tcPath.appendingPathComponent("clang").path - newEnv["CXX"] = tcPath.appendingPathComponent("clang++").path - return newEnv } @@ -288,6 +287,9 @@ extension Platform { return bin } + public func findToolchainBinDir(_ toolchain: ToolchainVersion) -> URL { + self.findToolchainLocation(toolchain).appendingPathComponent("usr/bin") + } #endif } diff --git a/Tests/SwiftlyTests/RunTests.swift b/Tests/SwiftlyTests/RunTests.swift index 846509c1..47337fd0 100644 --- a/Tests/SwiftlyTests/RunTests.swift +++ b/Tests/SwiftlyTests/RunTests.swift @@ -44,15 +44,8 @@ final class RunTests: SwiftlyTests { // The toolchains directory should be the fist entry on the path var run = try self.parseCommand(Run.self, ["run", try await Swiftly.currentPlatform.getShell(), "-c", "echo $PATH"]) var output = try await run.runWithMockedIO() + XCTAssert(output.count == 1) XCTAssert(output[0].contains(Swiftly.currentPlatform.swiftlyToolchainsDir.path)) - - // The CC and CXX variables should be set to clang/clang++ in the toolchains - run = try self.parseCommand(Run.self, ["run", try await Swiftly.currentPlatform.getShell(), "-c", "echo $CC; echo $CXX"]) - output = try await run.runWithMockedIO() - XCTAssert(output[0].hasPrefix(Swiftly.currentPlatform.swiftlyToolchainsDir.path)) - XCTAssert(output[0].hasSuffix("clang")) - XCTAssert(output[1].hasPrefix(Swiftly.currentPlatform.swiftlyToolchainsDir.path)) - XCTAssert(output[1].hasSuffix("clang++")) } } From bb36de02482d3bba733ba8a8f1d14906c42d2995 Mon Sep 17 00:00:00 2001 From: "Chris (SPG) McGee" Date: Tue, 5 Nov 2024 18:22:23 -0500 Subject: [PATCH 26/26] Fix the design document --- DESIGN.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/DESIGN.md b/DESIGN.md index 6bf507e0..f3572258 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -67,7 +67,7 @@ A simple setup for managing the toolchains could look like this: The toolchains (i.e. the contents of a given Swift download tarball) would be contained in the toolchains directory, each named according to the major/minor/patch version. `config.json` would contain any required metadata (e.g. the latest Swift version, which toolchain is selected, etc.). If pulling in Foundation to use `JSONEncoder`/`JSONDecoder` (or some other JSON tool) would be a problem, we could also use something simpler. -The `~/.local/bin` directory would include symlinks pointing to swiftly itself. When the proxies binaries are executed swiftly proxies them to the requested toolchain, or the default. +The `~/.local/share/swiftly/bin` directory would include symlinks pointing to swiftly itself. When the proxies binaries are executed swiftly proxies them to the requested toolchain, or the default. This is all very similar to how rustup does things, but I figure there's no need to reinvent the wheel here.