Swiftly proxies #155
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -4,21 +4,28 @@ This document contains the high level design of swiftly. Not all features have b | |||||
|
|
||||||
| ## Index | ||||||
|
|
||||||
| - [Swiftly's purpose](#swiftlys-purpose) | ||||||
| - [Installation of swiftly](#installation-of-switly) | ||||||
| - [Linux](#linux) | ||||||
| - [Installation of a Swift toolchain](#installation-of-a-swift-toolchain) | ||||||
| - [macOS](#macos) | ||||||
| - [Installation of a Swift toolchain](#installation-of-a-swift-toolchain-1) | ||||||
| - [Interface](#interface) | ||||||
| - [Toolchain names and versions](#toolchain-names-and-versions) | ||||||
| - [Commands](#commands) | ||||||
| - [Toolchain selection](#toolchain-selection) | ||||||
| - [Detailed design](#detailed-design) | ||||||
| - [Implementation sketch - Core](#implementation-sketch---core) | ||||||
| - [Implementation sketch - Ubuntu 20.04](#implementation-sketch---ubuntu-2004) | ||||||
| - [Implementation sketch - macOS](#implementation-sketch---macos) | ||||||
| - [`config.json` schema](#configjson-schema) | ||||||
|
|
||||||
| ## Swiftly's purpose | ||||||
|
|
||||||
| Swiftly helps you to easily install different Swift toolchains locally on your account. It also provides a single path where you can run the tools in the currently selected toolchain. Toolchain selection is [configurable](#toolchain-selection) using different mechanisms. | ||||||
|
|
||||||
| Note that swiftly is *not* a virtual toolchain in itself since there are cases where it cannot behave as a self-contained Swift toolchain. For example, there can be external dependencies on specific files, such as headers or libraries. There are far too many files that change between toolchain versions to be managed by swiftly. Also, for long-lived processes, there is no way to gracefully restart them without help from the client. | ||||||
|
|
||||||
| ## Installation of swiftly | ||||||
|
|
||||||
| The installation of swiftly is divided into two phases: delivery and initialization. Delivery of the swiftly binary can be accomplished using different methods: | ||||||
|
|
@@ -60,7 +67,7 @@ A simple setup for managing the toolchains could look like this: | |||||
|
|
||||||
| The toolchains (i.e. the contents of a given Swift download tarball) would be contained in the toolchains directory, each named according to the major/minor/patch version. `config.json` would contain any required metadata (e.g. the latest Swift version, which toolchain is selected, etc.). If pulling in Foundation to use `JSONEncoder`/`JSONDecoder` (or some other JSON tool) would be a problem, we could also use something simpler. | ||||||
|
|
||||||
| The `~/.local/bin` directory would include symlinks pointing to swiftly itself. When the proxies binaries are executed swiftly proxies them to the requested toolchain, or the default. | ||||||
|
|
||||||
| This is all very similar to how rustup does things, but I figure there's no need to reinvent the wheel here. | ||||||
|
|
||||||
|
|
@@ -78,7 +85,7 @@ The contents of `~/Library/Application Support/swiftly` would look like this: | |||||
| – env | ||||||
| ``` | ||||||
|
|
||||||
| Instead of downloading tarballs containing the toolchains and storing them directly in `~/.local/share/swiftly/toolchains`, we instead install Swift toolchains to `~/Library/Developer/Toolchains` via the `.pkg` files provided for download at swift.org. In the env file, we’ll add a line that looks like `export PATH="$HOME/Library/Application Support/swiftly:$PATH"`, so that swiftly can proxy toolchain commands to the requested toolchain, or default. `config.json` will contain version information about the selected toolchain as well as its actual location on disk. | ||||||
|
|
||||||
| This scheme works for ensuring the version of Swift used on the command line can be controlled, but it doesn’t affect the active toolchain used by Xcode, which uses its own mechanisms for that. Xcode, if it is installed, can find the toolchains installed by swiftly. | ||||||
|
|
||||||
|
|
@@ -138,6 +145,14 @@ Installing a specific snapshot from a swift version development branch | |||||
|
|
||||||
| `swiftly install 5.5-snapshot-2022-1-28` | ||||||
|
|
||||||
| ##### Installing the version from the `.swift-version` file | ||||||
|
|
||||||
| A package could have a swift version file that specifies the recommended toolchain version. A swiftly install with no version will search for a version file and install that version. | ||||||
|
|
||||||
| `swiftly install` | ||||||
|
|
||||||
| If no swift version file can be found then the installation fails indicating that it couldn't fine the file. | ||||||
|
||||||
|
|
||||||
| #### uninstall | ||||||
|
|
||||||
| Uninstalling versions of Swift should be in a similar form to install. Uninstalling a toolchain that is currently “in use” (see the “use” command section below) will cause swiftly to use the latest Swift release toolchain that is installed. If none are, the latest snapshot will be used. If no snapshots are installed either, then a message will be printed indicating that all Swift versions are uninstalled. | ||||||
|
|
@@ -178,7 +193,7 @@ To list all the versions of swift installed on your system | |||||
|
|
||||||
| #### use | ||||||
|
|
||||||
| “Using” a toolchain sets it as the default toolchain, meaning it will be the default one that is used when running toolchain commands from the shell. Only a single toolchain can be the default at a given time and location. Using a toolchain doesn’t uninstall anything; it only updates the configuration. | ||||||
|
|
||||||
| To use the toolchain associated with the most up-to-date Swift version, the “latest” version can be specified: | ||||||
|
|
||||||
|
|
@@ -208,6 +223,10 @@ To use the latest installed main snapshot, leave off the date: | |||||
|
|
||||||
| `swiftly use main-snapshot` | ||||||
|
|
||||||
| The use subcommand also supports `.swift-version` files. If version file is present in the current working directory, or an ancestory directory, then swiftly will update that file with the new version to use. This can be a useful feature for a team to share and align on toolchain versions with git. As a special case, if swiftly could not find a version file, but it could find a Package.swift file it will create a new version file for you in the package and set that to the requested toolchain version. | ||||||
|
||||||
| The use subcommand also supports `.swift-version` files. If version file is present in the current working directory, or an ancestory directory, then swiftly will update that file with the new version to use. This can be a useful feature for a team to share and align on toolchain versions with git. As a special case, if swiftly could not find a version file, but it could find a Package.swift file it will create a new version file for you in the package and set that to the requested toolchain version. | |
| If a .swift-version file |
Simple wording change suggestion
There was a problem hiding this comment.
I would rather the default is the other way. ie use edits global settings and you can change local settings with a --local flag.
There was a problem hiding this comment.
I think that there are advantage of having this local as the default. It encourages the sharing of the toolchain revision that the team is working with through discovery of this feature of swiftly. Also, this helps to tie into one of the values of using swiftly itself, which is how easily and flexibly you can switch toolchains.
In terms of precedence, git defaults to local over global for its configuration.
I'm curious what you think are the pitfalls of this approach.
There was a problem hiding this comment.
Yeah I realised as I was writing this, there is previous art that defaults to local. It is more a preference on my behalf and how I'd use swiftly
There was a problem hiding this comment.
pyenv and rbenv call this command local, global or shell depending on whether you're setting the Python/Ruby version for a directory, globally on the machine (for your user), or just in the current shell. The shell setting overrides the local setting, which overrides the global setting.
Outdated
There was a problem hiding this comment.
Personally, I don't think manipulating PATH is a good idea. Swiftly should have its proxy bin directory added to the path when it is installed, but should refrain from messing with PATH thereafter. Similarly for CC and CXX.
There was a problem hiding this comment.
I've removed the setting of CC and CXX variables, opting instead to have the user provide those if they need them, copied verbatim along with the rest of their environment. The only environment changes is to the PATH prepending the current toolchain, and leaving the rest of it untouched as the user had set previously.
Overriding the proxies in this case has the benefit of keeping this as a workaround for proxy performance issues. Also, this approach has pyenv as a precedent, which is documented to be doing the exact same thing. https://github.com/pyenv/pyenv/blob/master/COMMANDS.md#pyenv-exec
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO we ought to be using
~/.swiftly/binfor this. Swiftly doesn't own that directory and while it's fine to install theswiftlycommand itself there (assuming it's being installed per-user rather than system-wide), I think it's very bad form installing "proxies" in there.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, I've changed this to put the proxies and swiftly itself into its own managed directory by default, which is
~/.local/share/swiftly/bin. This matches what we do on macOS, except there we use a more typical macOS location~/Library/Application Support/swiftly/bin.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
~/.local/binwas originally chosen according to the XDG Base Directories specification, which is something of a standard for directories like this. While swiftly doesn't own~/.local/bin, it is the designated place to drop per-user executables according to that spec. Why would it be bad form to put our symlinks there?https://specifications.freedesktop.org/basedir-spec/latest/
https://wiki.archlinux.org/title/XDG_Base_Directory
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The concern is that the user might have swift toolchain, and other items (clang, lld, lldb, etc.) existing in the ~/.local/bin directory and swiftly will attempt to manage those.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are really two reasons for this; the first is that, as @cmcgee1024 says, you might actually have Swift installed in
~/.local/bin, in which case the proxies would then overwrite your Swift executables, which would be wrong. The second is that when using tools likeswiftly, there are situations where you want to bypass the proxies, and it's easiest to do that if they're in a separatebindirectory all of their own (since whether you're using the proxies or not is dependent on whether thatbindirectory appears in yourPATH).There is precedent here;
pyenv,rbenvet al also put their shims (which we seem to be calling proxies) into a separatebindirectory for the same reasons.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think
swiftlyitself belongs in~/.local/binor/usr/local/binor wherever the user installed it. It's just the proxies that should live elsewhere IMO. // @cmcgee1024There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@al45tair swiftly is aware of two locations at the moment. There is the swiftly home dir, where its configuration lives, and the swiftly bin dir where all of the binaries that it manages live. I'm hesitant to add another location to the list.
In the Linux case
~/.local/bindoesn't tend to be on the user's path, so swiftly finds itself adding it as part of augmenting the user's profile, being mindful of collisions with other packages that might attempt to do the same, or even the user themselves. There's no similar common CLT bin directory at a user level for macOS, so the swiftly bin directory is~/Application Support/swiftly/binas the default. There's a far less likelihood of a path collision where something else that adds swiftly's managed bin directory. I think it makes sense to keep all of the binaries that swiftly manages (potentially itself too), separate from a shared binary location on all platforms as the default. The user can customize the paths if they want in the installer to change this, but then they are taking on more responsibility for the possibility of collisions.Swiftly itself can be installed by a system package manager where it will probably be installed in one of the usual locations like
/usr/binor/usr/local/bin. The latter is where swiftly is installed using the macOS pkg file. The package manager manages these paths and the user's profile likely includes them in their path without extra handling by swiftly. In this case the swiftly bin directory is just the proxies that swiftly manages since it can't manage its own binary.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't have a strong objection to that, I suppose, but it does mean that you can't use
swiftlywithout having the proxies in your path if you've letswiftlyinstall itself. At least, unless you make a symlink from~/.local/binor similar.