Swiftly proxies #155

cmcgee1024 · 2024-08-10T12:17:27Z

Add a design for the new swiftly proxy system for more flexible routing of command invocations to a toolchain.

Re-target the symbolic links that swiftly creates to point to swiftly itself, which can decide the selected toolchain more flexibly on a per-invocation basis.

Add support for reading the toolchain version from a .swift-version file if a selector isn't given. The swiftly use command will attempt to update a swift version file, or in certain circumstances it will create the version file for you. The special swiftly install with no selector argument that will attempt to install a toolchain version specified in a version file.

Create a swiftly run subcommand that can run arbitrary commands, such as build scripts or makefile generators. This subcommand will set the PATH environment variable to the currently selected toolchain, and other special environment variables like CC and CXX to specifically point to the clang and clang++ executables. Add an ability for this subcommand to accept arbitrary toolchain selectors that take the higher precedence over the presence of the swift version file or the global default.

swiftly run swift build +latest
swiftly run cmake -G ninja +main-snapshot

Update swiftly use and swiftly list to take into account that there is a global default, which is the fallback if there is no .swift-version file, but also a potentially different toolchain that is in use in the current context. The swiftly use subcommand with not arguments will show in brackets the reason why the current in-use toolchain is selected, whether it is a special .swift-version file in which case it will show the file path, or it is the global default.

Add a design proposal for the new swiftly proxy system

…t-version file

BrianHenryIE · 2024-08-18T00:35:19Z

Hijackingswift altogether isn't ideal. I've used which swift to determine the toolchain directory and proxying adds a level of indirection that isn't obvious.

https://github.com/anders0nmat/swift-embedded-pm/pull/3/files

Consider:

swiftly run 5.10.1 build

Or proxy CLI arguments that don't exist in swiftly to swift:

swiftly build

(I just write Swift for fun, so take my suggestions with a grain of salt)

cmcgee1024 · 2024-08-20T12:28:53Z

@BrianHenryIE your feedback is appreciated, thank you.

I've used which swift to determine the toolchain directory and proxying adds a level of indirection that isn't obvious.

https://github.com/anders0nmat/swift-embedded-pm/pull/3/files

It seems from the above patch that the end goal is to be able to invoke the llvm-ar executable from the current toolchain. I'm curious, why dig through the symlink to the real path using the realpath command? The llvm-ar command should be a sibling of the swift command, whether if it is a symlink created by swiftly, or a toolchain binary installed by other means (e.g. manual installation, system package). Inspecting the symlink would seem to couple this CMake script to swiftly being the toolchain manager.

Consider:

swiftly run 5.10.1 build

Providing a swiftly run subcommand is something that we can consider as an enhancement to the current swiftly design. We could allow someone to run any command that they want, not just the toolchain ones. The command runs where the PATH is set to the toolchain binaries with the highest precedence. Also, we can set certain environment variables like CC and CXX to the selected clang version.

# Let's use the swift toolchain to compile this C project:
swiftly run 5.10 ./configure
swiftly run 5.10 make

We can put these ideas and more into #22

adam-fowler

I'm not 100% convinced by this yet. I wonder how the core team would feel about the swift exe being hijacked by swiftly.

adam-fowler · 2024-08-20T12:36:56Z

DESIGN.md


+The use subcommand also supports `.swift-version` files. If version file is present in the current working directory, or an ancestory directory, then swiftly will update that file with the new version to use. This can be a useful feature for a team to share and align on toolchain versions with git. As a special case, if swiftly could not find a version file, but it could find a Package.swift file it will create a new version file for you in the package and set that to the requested toolchain version.
+
+Note: The `.swift-version` file mechanisms can be overridden using the `--global-default` flag so that your swiftly installation's default toolchain can be set explicitly.


I would rather the default is the other way. ie use edits global settings and you can change local settings with a --local flag.

I think that there are advantage of having this local as the default. It encourages the sharing of the toolchain revision that the team is working with through discovery of this feature of swiftly. Also, this helps to tie into one of the values of using swiftly itself, which is how easily and flexibly you can switch toolchains.

In terms of precedence, git defaults to local over global for its configuration.

I'm curious what you think are the pitfalls of this approach.

Yeah I realised as I was writing this, there is previous art that defaults to local. It is more a preference on my behalf and how I'd use swiftly

pyenv and rbenv call this command local, global or shell depending on whether you're setting the Python/Ruby version for a directory, globally on the machine (for your user), or just in the current shell. The shell setting overrides the local setting, which overrides the global setting.

DESIGN.md

cmcgee1024 · 2024-08-20T12:49:01Z

I'm not 100% convinced by this yet. I wonder how the core team would feel about the swift exe being hijacked by swiftly.

I'm definitely interested to hear if there are opinions from the core team about this enhancement. I prefer to think of this as extending the semantics, not "hijacking" the toolchain itself. There are precedents for a proxy mechanism of a toolchain manager on top of the toolchain itself, such as rustup.

adam-fowler · 2024-08-20T13:25:27Z

DESIGN.md

+* The presence of a .swift-version file in the current working directory, or ancestor directory, with the required toolchain version
+* The swiftly default (in-use) toolchain set in the config.json by `swiftly install` or `swiftly use` commands
+
+In the first two cases, if there is no matching toolchain installed, swiftly will attempt to automatically install the requested toolchain and use it if the installation succeeeds.


We'll need a way to disable this. People with a bad network connection will not be wanting to download 900MB of swift toolchain just to build a project.

Yes, I'll update this since this has come up a few times. It could be a case of over automation. Instead, if there's no matching toolchain then the command fails. The swiftly install with no arguments can be used to install the currently selected toolchain with the full capabilities of that subcommand, including post install script, etc. Also, it's clear that an error is related to the install operation itself since that is the request.

Could we also have a command line option to ignore the .swift-version file. This would make it easier for things like CI to test a library (which includes a .swift-version file) against multiple swift toolchains

Yes, I think that this in this case the command-line selectors will take precedence over both the .swift-version file and the global default in-use toolchain from the config.json. The CI can do this to force specific toolchain(s):

swiftly install 5.10.1 # Install this specific toolchain swift build +5.10.1 # Build with this toolchain

…ocation clarify the boundaries of the swiftly toolchain abstraction and elaborate on how to work around them

0xTim

I'd be interested to hear the core team's view on intercepting any calls to swift. Node does a similar thing with nvm IIRC so there's precedent, not that that's the best example of an ecosystem if there are any security concerns 😅

I would say that the + syntax is not overly discoverable or intuitive, I'm not aware of any other tool that has this. I'd be more inclined to have an actual flag that can either be intercepted or called via swiftly run etc.

Overall however I agree that having a mechanism for being able to run commands in Swift with different versions without having to constantly switch your local version is valuable and should be implemented

0xTim · 2024-08-23T15:15:29Z

DESIGN.md

+
+Swiftly helps you to easily install different Swift toolchains locally on your account. It also provides a single path where you can run the tools in the currently selected toolchain. Toolchain selection is [configurable](#toolchain-selection) using different mechanisms.
+
+Note that swiftly is *not* a virtual toolchain in itself since there are cases where it cannot behave as a self-contained Swift toolchain. For example, there can be external dependencies on specific files, such as headers or libraries, far too many and variable between toolchain versions to be managed by swiftly. Also, for long-lived processes, there is no way to gracefully restart them without help from the client.


This sentence needs some tweaking I think

Suggested change

Note that swiftly is *not* a virtual toolchain in itself since there are cases where it cannot behave as a self-contained Swift toolchain. For example, there can be external dependencies on specific files, such as headers or libraries, far too many and variable between toolchain versions to be managed by swiftly. Also, for long-lived processes, there is no way to gracefully restart them without help from the client.

Note that swiftly is *not* a virtual toolchain in itself since there are cases where it cannot behave as a self-contained Swift toolchain. For example, there can be external dependencies on specific files, such as headers or libraries, far too many variables between toolchain versions to be managed by swiftly. Also, for long-lived processes, there is no way to gracefully restart them without help from the client.

cmcgee1024 · 2024-08-26T12:14:01Z

I'd be interested to hear the core team's view on intercepting any calls to swift. Node does a similar thing with nvm IIRC so there's precedent, not that that's the best example of an ecosystem if there are any security concerns 😅

Do you know a way to engage the core team on this? I'm also curious if there are any concerns or suggestions from there. The proxy idea has been mentioned in the forums a couple of times now, the last one points to this PR. I have received some feedback, but I don't know if any have been representative of the core.

I would say that the + syntax is not overly discoverable or intuitive, I'm not aware of any other tool that has this. I'd be more inclined to have an actual flag that can either be intercepted or called via swiftly run etc.

Thank you, this is a fair observation.

The proxy concept along with the + syntax is inspired in part by rustup:
=> https://rust-lang.github.io/rustup/concepts/proxies.html

In their documentation there is only a small mention of the special selector syntax:
=> https://rust-lang.github.io/rustup/overrides.html#toolchain-override-shorthand

I think that this syntax can be discoverable in swiftly by providing (some) examples in the getting started guide, demonstrating how to use swiftly. In the swiftly steady state I imagine that the syntax won't be used very often as the toolchain will likely be selected with either the in-use global default for the user, or hopefully, the project's .swift-version file that aligns the project participants. In these cases swiftly works just like using the toolchain directly, with the exception of times when the selected toolchain isn't installed where it prompts the user to install it. This is what's interesting about the proxy concept.

I can see the + syntax being particularly useful when someone wants to run a check against the latest snapshot, or a previous stable build.

Overall however I agree that having a mechanism for being able to run commands in Swift with different versions without having to constantly switch your local version is valuable and should be implemented

I'm wondering, since this has come up a few times, if it might be worth providing both proxy capability and #22 at the same time in this PR to suit different preferences and use cases.

patrickfreed

I personally tend to prefer a subcommand over the + syntax since swiftly isn't ubiquitous (yet 🙂) like rustup is, and so introducing swiftly-specific syntax to a non-swiftly binary invocation may cause confusion for non-swiftly users when they encounter someone else using it (or more frustratingly, compatibility issues in scripts and what not). That said, it is a generally useful syntax, so if the core team is supportive, then I don't have a problem with it.

To avoid blocking the work for this, I'd suggest we start with the more conservative swiftly run <toolchain> or swiftly exec <toolchain> command and then add the + syntax later.

patrickfreed · 2024-08-26T14:24:21Z

DESIGN.md

+For cases where the physical toolchain must be located, such as references specific header files, or shared libraries that are not proxied by swiftly there is a method to resolve the currently selected toolchain to its physical location using `swiftly use`.
+
+```
+swiftly use --location


perhaps --print-location? As it is, it looks like --location affects use in some way.

patrickfreed · 2024-08-26T14:25:24Z

DESIGN.md

 The toolchains (i.e. the contents of a given Swift download tarball) would be contained in the toolchains directory, each named according to the major/minor/patch version. `config.json` would contain any required metadata (e.g. the latest Swift version, which toolchain is selected, etc.). If pulling in Foundation to use `JSONEncoder`/`JSONDecoder` (or some other JSON tool) would be a problem, we could also use something simpler.

-The `~/.local/bin` directory would include symlinks pointing to the `bin` directory of the "active" toolchain, if any.
+The `~/.local/bin` directory would include symlinks pointing to swiftly itself. When these binaries are run swiftly proxies them to the requested toolchain, or a default.


I think it would be good to more clearly explain what "these binaries" refer to in this sentence.

0xTim · 2024-08-27T12:55:31Z

Do you know a way to engage the core team on this?

Maybe @shahmishal can bring it up but this should probably be raised at the Platform Steering Group until whatever group will end up with ownership of this takes over them

airspeedswift · 2024-08-27T16:22:10Z

this should probably be raised at the Platform Steering Group

I think this is right. There are plans to add a third steering group in future (dubbed "Ecosystem") and this might fall under that rather than Platform, but I could see it going either way in this case so starting with the platform steering group is the right way to go for now.

The first step is to write up a pitch and post it to the forums. That pitch should cover the pros/cons of the approach, and recommend one (with the others being alternatives considered). Then engagement on the merits can happen in that thread. That said, it's fine to hash out early ideas on a doc in this repo beforehand.

Once the pitch has had some time to converge on a way forward, the platform steering group would take it up as a proposal. You don't need the steering group to pre-clear that pitch, though you might want to give them a heads up about it, so they can engage in it too since they'll ultimately be the ones deciding which way to go with this.

Note that since Swiftly was ingested into swiftlang and so is part of the Swift project now, I don't see anything contentious about close integration between the swift launch tool and swiftly per se. But of course, the other tradeoffs cited in this thread about exactly how that integration actually works should be discussed.

cmcgee1024 · 2024-08-28T16:04:51Z

Thanks everyone for the discussion here. This has been really positive. I will change the shape of this PR in a couple of ways and see if this can be accomplished in the near term:

Retain proxy concept for further refinement in the future.
Implement Implement exec command #22 in here to support a swiftly specific syntax for on-the-fly toolchain selection and also the extra capabilities that feature can provide.
Defer the special toolchain selector syntax (ie. the injection of the '+' into tool invocations) for later after a pitch has been made on the forums and the platform steering group.

Please let me know if there are any concerns over this latest pivot.

…new swiftly run command

Change the nature of the swiftly symlinks so that they point to the swiftly executable at install time. These do not change when new toolchains are used. Toolchain selection happens each time when the proxies are run. The proxies are created for a well-known set of toolchain binaries that are constant for a wide variety of toolchain versions and platforms. Add support for .swift-version files for toolchain selection. Update the use command so that it can point out which toolchain is in use based on context, such as swift version files that are located in the current working directory or above. The fallback selection comes from the global default configuration's 'inUse' setting. When querying for what's in use the global default is shown with the "(default)" tag. If the in-use toolchain is selected by a swift-version file the path to that file is displayed. Provide a print location flag to the use subcommand that can print the file path of the toolchain that is in use in the current context. When using a new toolchain, depending on whether a swift version is selecting the current one, update the swift version file with the selected toolchain version. If no swift version file can be located, attempt to create a new one at the top of the git worktree. If there is no git worktree, then fallback to updating the global default in the configuration. Provide a global default flag for the use subcommand so that only the global default in-use toolchain is considered and not any of the swift version files.

cmcgee1024 · 2024-09-04T20:04:53Z

@swift-ci test macOS

rauhul · 2024-09-05T16:00:48Z

Out of curiosity, do you have an idea of how this would interact with xcrun?

cmcgee1024 · 2024-09-05T18:10:13Z

Out of curiosity, do you have an idea of how this would interact with xcrun?

Swiftly's mechanisms for toolchain selection are entirely separate from xcrun. But, it will install toolchain packages in the user's volume, and those should be reachable with xcrun.

…esult and centralized error messages

Provide a run command that allows arbitrary commands to be run in the context of the selected toolchain. Provide a one-off selection mechanism with a special syntax on the run command.

cmcgee1024 · 2024-09-06T20:01:34Z

@swift-ci test macOS

justice-adams-apple · 2024-10-17T22:13:54Z

DESIGN.md

+
+`swiftly install`
+
+If no swift version file can be found then the installation fails indicating that it couldn't fine the file.


Very small nit, but could we use

If no .swift-version file ...

Rather than

swift version file

Just to be consistent, so users don't conflate the "swift version file" with something else in the Swift ecosystem. This way we explicitly mention the file in question. This is how it's written in the "Installing the version from the .swift-version files" section

justice-adams-apple · 2024-10-17T22:13:58Z

DESIGN.md


 `swiftly use main-snapshot`

+The use subcommand also supports `.swift-version` files. If version file is present in the current working directory, or an ancestory directory, then swiftly will update that file with the new version to use. This can be a useful feature for a team to share and align on toolchain versions with git. As a special case, if swiftly could not find a version file, but it could find a Package.swift file it will create a new version file for you in the package and set that to the requested toolchain version.


Suggested change

The use subcommand also supports `.swift-version` files. If version file is present in the current working directory, or an ancestory directory, then swiftly will update that file with the new version to use. This can be a useful feature for a team to share and align on toolchain versions with git. As a special case, if swiftly could not find a version file, but it could find a Package.swift file it will create a new version file for you in the package and set that to the requested toolchain version.

If a .swift-version file

Simple wording change suggestion

justice-adams-apple · 2024-10-17T22:14:05Z

Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md

    $ swiftly install 5.7-snapshot
    $ swiftly install main-snapshot

+ Install whatever toolchain is current selected, such as a .swift-version file:


Suggested change

Install whatever toolchain is current selected, such as a .swift-version file:

Install whatever toolchain is currently selected, such as a .swift-version file:

justice-adams-apple · 2024-10-17T22:14:10Z

Documentation/SwiftlyDocs.docc/swiftly-cli-reference.md

+*Run a command while proxying to the selected toolchain commands.*
+
+
+Run a command with a selected toolchain, so that all toolchain commands are become the default added to the system path and other common environment variables.


The wording of this sentence confuses me a bit, might just need some reworking

justice-adams-apple · 2024-10-17T22:14:17Z

Sources/Swiftly/Install.swift

            $ swiftly install 5.7-snapshot
            $ swiftly install main-snapshot
+
+         Install whatever toolchain is current selected, such as a .swift-version file:


Suggested change

Install whatever toolchain is current selected, such as a .swift-version file:

Install whatever toolchain is currently selected, such as a .swift-version file:

justice-adams-apple · 2024-10-17T22:14:25Z

Sources/Swiftly/Uninstall.swift

+                    try await Use.execute(toUse, globalDefault: true, &config)
                } else {
                    // If there are no more toolchains installed, just unuse the currently active toolchain.
-                    try Swiftly.currentPlatform.unUse(currentToolchain: toolchain)


Do we still want to unuse the toolchain if there are no more toolchains to be set active, in order to clean up all the symlinks?

With this change the proxy symlinks are created at install time of swiftly and remain there. Keeping them stable helps with certain shells that aggressively cache commands on the PATH, such as zsh. If there are no toolchains installed and someone tries to run "swift", "clang", or some other toolchain command then swiftly will give an error indicating that they can install one.

interesting, so by installing swiftly then you wouldn't really be able to invoke your original swift toolchain installation?

Say I had a toolchain installed, then I decide to install swiftly, I download some toolchains with swiftly, then ultimately uninstall them all. I then go to invoke swift again under the assumption my default toolchain which was installed prior to swiftly would take precedence. Sounds like in this case that would not work because all the symlinks would still be proxying through swiftly which wouldn't be able to find a toolchain?

Right, by installing swiftly you're giving it the ability to control which toolchain you are invoking. Note that since swiftly maintains its own record of toolchains it installed, any that you've installed previously wouldn't be known to swiftly since they could be in virtually any location on the system.

The user can take control temporarily by overriding their PATH to the external toolchain that they want to use, or uninstall swiftly.

Fix symlink target selection for swiftly when it is system managed

cmcgee1024 · 2024-10-18T12:44:05Z

@swift-ci test macOS

cmcgee1024 · 2024-10-18T13:06:05Z

Note that macOS CI is failing in a way that is unrelated to the tests. The tests are all green:

Test Suite 'All tests' passed at 2024-10-18 12:48:47.119.
	 Executed 67 tests, with 0 failures (0 unexpected) in 78.244 (78.252) seconds.

Print the error in a better way

cmcgee1024 · 2024-10-21T10:55:53Z

@swift-ci test macOS

cmcgee1024 · 2024-10-23T19:06:40Z

@swift-ci test macOS

al45tair · 2024-11-04T16:16:55Z

DESIGN.md

-The `~/.local/bin` directory would include symlinks pointing to the `bin` directory of the "active" toolchain, if any.
+The `~/.local/bin` directory would include symlinks pointing to swiftly itself. When the proxies binaries are executed swiftly proxies them to the requested toolchain, or the default.


IMO we ought to be using ~/.swiftly/bin for this. Swiftly doesn't own that directory and while it's fine to install the swiftly command itself there (assuming it's being installed per-user rather than system-wide), I think it's very bad form installing "proxies" in there.

Thanks, I've changed this to put the proxies and swiftly itself into its own managed directory by default, which is ~/.local/share/swiftly/bin. This matches what we do on macOS, except there we use a more typical macOS location ~/Library/Application Support/swiftly/bin.

~/.local/bin was originally chosen according to the XDG Base Directories specification, which is something of a standard for directories like this. While swiftly doesn't own ~/.local/bin, it is the designated place to drop per-user executables according to that spec. Why would it be bad form to put our symlinks there?

https://specifications.freedesktop.org/basedir-spec/latest/
https://wiki.archlinux.org/title/XDG_Base_Directory

The concern is that the user might have swift toolchain, and other items (clang, lld, lldb, etc.) existing in the ~/.local/bin directory and swiftly will attempt to manage those.

There are really two reasons for this; the first is that, as @cmcgee1024 says, you might actually have Swift installed in ~/.local/bin, in which case the proxies would then overwrite your Swift executables, which would be wrong. The second is that when using tools like swiftly, there are situations where you want to bypass the proxies, and it's easiest to do that if they're in a separate bin directory all of their own (since whether you're using the proxies or not is dependent on whether that bin directory appears in your PATH).

There is precedent here; pyenv, rbenv et al also put their shims (which we seem to be calling proxies) into a separate bin directory for the same reasons.

Thanks, I've changed this to put the proxies and swiftly itself into its own managed directory by default

I think swiftly itself belongs in ~/.local/bin or /usr/local/bin or wherever the user installed it. It's just the proxies that should live elsewhere IMO. // @cmcgee1024

@al45tair swiftly is aware of two locations at the moment. There is the swiftly home dir, where its configuration lives, and the swiftly bin dir where all of the binaries that it manages live. I'm hesitant to add another location to the list.

In the Linux case ~/.local/bin doesn't tend to be on the user's path, so swiftly finds itself adding it as part of augmenting the user's profile, being mindful of collisions with other packages that might attempt to do the same, or even the user themselves. There's no similar common CLT bin directory at a user level for macOS, so the swiftly bin directory is ~/Application Support/swiftly/bin as the default. There's a far less likelihood of a path collision where something else that adds swiftly's managed bin directory. I think it makes sense to keep all of the binaries that swiftly manages (potentially itself too), separate from a shared binary location on all platforms as the default. The user can customize the paths if they want in the installer to change this, but then they are taking on more responsibility for the possibility of collisions.

Swiftly itself can be installed by a system package manager where it will probably be installed in one of the usual locations like /usr/bin or /usr/local/bin. The latter is where swiftly is installed using the macOS pkg file. The package manager manages these paths and the user's profile likely includes them in their path without extra handling by swiftly. In this case the swiftly bin directory is just the proxies that swiftly manages since it can't manage its own binary.

I don't have a strong objection to that, I suppose, but it does mean that you can't use swiftly without having the proxies in your path if you've let swiftly install itself. At least, unless you make a symlink from ~/.local/bin or similar.

al45tair · 2024-11-04T16:24:54Z

DESIGN.md

+swiftly run make
+```
+
+Swiftly adjusts certain environment variables, such as prefixing the PATH to the selected toolchain directory, and setting the CC and CXX variables to the locations of clang and clang++ in those toolchains so that the build tools use them. If you want to explicitly specify a toolchain for the command you can do that with a selector notation like this:


Personally, I don't think manipulating PATH is a good idea. Swiftly should have its proxy bin directory added to the path when it is installed, but should refrain from messing with PATH thereafter. Similarly for CC and CXX.

I've removed the setting of CC and CXX variables, opting instead to have the user provide those if they need them, copied verbatim along with the rest of their environment. The only environment changes is to the PATH prepending the current toolchain, and leaving the rest of it untouched as the user had set previously.

Overriding the proxies in this case has the benefit of keeping this as a workaround for proxy performance issues. Also, this approach has pyenv as a precedent, which is documented to be doing the exact same thing. https://github.com/pyenv/pyenv/blob/master/COMMANDS.md#pyenv-exec

al45tair · 2024-11-04T16:27:41Z

DESIGN.md


+The use subcommand also supports `.swift-version` files. If version file is present in the current working directory, or an ancestory directory, then swiftly will update that file with the new version to use. This can be a useful feature for a team to share and align on toolchain versions with git. As a special case, if swiftly could not find a version file, but it could find a Package.swift file it will create a new version file for you in the package and set that to the requested toolchain version.
+
+Note: The `.swift-version` file mechanisms can be overridden using the `--global-default` flag so that your swiftly installation's default toolchain can be set explicitly.


pyenv and rbenv call this command local, global or shell depending on whether you're setting the Python/Ruby version for a directory, globally on the machine (for your user), or just in the current shell. The shell setting overrides the local setting, which overrides the global setting.

… ones, giving a message about the shell Set only the PATH environment in swiftly run leaving CC and CXX to the user

cmcgee1024 · 2024-11-05T23:20:51Z

@swift-ci test macOS

cmcgee1024 · 2024-11-05T23:29:37Z

@swift-ci test macOS

cmcgee1024 · 2024-11-08T20:30:33Z

@swift-ci test macOS

cmcgee1024 · 2024-11-08T20:36:16Z

The macOS tests are passing:

Test Suite 'All tests' passed at 2024-11-08 20:35:02.246.
	 Executed 67 tests, with 0 failures (0 unexpected) in 75.112 (75.121) seconds

shahmishal · 2024-11-15T22:48:02Z

@swift-ci test macOS

shahmishal · 2024-11-15T23:43:54Z

@al45tair @justice-adams-apple When you get a change, can you review the latest changes?

cmcgee1024 · 2024-11-18T15:43:58Z

@swift-ci test macOS

cmcgee1024 · 2024-11-19T18:27:32Z

@swift-ci test macOS

justice-adams-apple

LGTM

al45tair

I haven't checked carefully through the code changes to swiftly itself, but I think the overall design seems fine. (I still have a slight reservation about the install location of the swiftly binary itself, but it's relatively minor and easily fixed by the user with a symlink if they care.)

cmcgee1024 added 2 commits August 10, 2024 08:16

DRAFT: Swiftly proxies

e4efbeb

Add a design proposal for the new swiftly proxy system

Add a swiftly install workflow where the version comes from the .swif…

3b5c404

…t-version file

cmcgee1024 requested review from adam-fowler and patrickfreed August 10, 2024 12:22

adam-fowler reviewed Aug 20, 2024

View reviewed changes

cmcgee1024 added 2 commits August 20, 2024 12:15

update design to make auto-installation an error instead

956256f

provide a mechanism to find the currently in-use toolchain physical l…

375df29

…ocation clarify the boundaries of the swiftly toolchain abstraction and elaborate on how to work around them

cmcgee1024 mentioned this pull request Aug 21, 2024

Implement exec command #22

Closed

add more details about the selector prefix, and methods to escape

ed68a9e

0xTim reviewed Aug 23, 2024

View reviewed changes

patrickfreed reviewed Aug 26, 2024

View reviewed changes

Restructure the PR to move the selector syntax from the proxies to a …

c745f4c

…new swiftly run command

cmcgee1024 mentioned this pull request Aug 30, 2024

Robust toolchain resolution for Code LLDB dylib swiftlang/vscode-swift#1045

Open

Merge branch 'main' of github.com:cmcgee1024/swiftly into proxies-design

ae29e88

cmcgee1024 added 3 commits September 5, 2024 15:09

Rewrite the select toolchain function with a type for the selection r…

ce0d27a

…esult and centralized error messages

Update the documentation

66dd459

Create a swiftly run command

2df7357

Provide a run command that allows arbitrary commands to be run in the context of the selected toolchain. Provide a one-off selection mechanism with a special syntax on the run command.

justice-adams-apple reviewed Oct 17, 2024

View reviewed changes

cmcgee1024 added 2 commits October 18, 2024 08:23

Make recommended documentation changes.

5e615ea

Fix symlink target selection for swiftly when it is system managed

Merge branch 'main' of github.com:cmcgee1024/swiftly into proxies-design

10a0856

cmcgee1024 added 2 commits October 19, 2024 07:58

Provide a better error message on swiftly install with no version

6d6050e

Print the error in a better way

Update README, and add documentation for the new run subcommand

6989796

cmcgee1024 added 2 commits October 23, 2024 11:20

Prompt before updating the .swift-version file.

52d081f

Merge branch 'main' of github.com:cmcgee1024/swiftly into proxies-design

2f6d701

al45tair reviewed Nov 4, 2024

View reviewed changes

Create proxies on toolchain installation, creating only the necessary…

cb0e923

… ones, giving a message about the shell Set only the PATH environment in swiftly run leaving CC and CXX to the user

Fix the design document

bb36de0

Merge branch 'main' of github.com:cmcgee1024/swiftly into proxies-design

7269128

Merge branch 'main' of github.com:cmcgee1024/swiftly into proxies-design

35cb5c5

cmcgee1024 requested review from al45tair and justice-adams-apple November 18, 2024 16:23

justice-adams-apple approved these changes Nov 20, 2024

View reviewed changes

al45tair approved these changes Nov 21, 2024

View reviewed changes

cmcgee1024 merged commit 0ce6cc5 into swiftlang:main Nov 21, 2024
20 checks passed

cmcgee1024 mentioned this pull request Nov 21, 2024

Swiftly proxies for tools in the toolchain #150

Closed


		The use subcommand also supports `.swift-version` files. If version file is present in the current working directory, or an ancestory directory, then swiftly will update that file with the new version to use. This can be a useful feature for a team to share and align on toolchain versions with git. As a special case, if swiftly could not find a version file, but it could find a Package.swift file it will create a new version file for you in the package and set that to the requested toolchain version.

		Note: The `.swift-version` file mechanisms can be overridden using the `--global-default` flag so that your swiftly installation's default toolchain can be set explicitly.


		Swiftly helps you to easily install different Swift toolchains locally on your account. It also provides a single path where you can run the tools in the currently selected toolchain. Toolchain selection is [configurable](#toolchain-selection) using different mechanisms.

		Note that swiftly is not a virtual toolchain in itself since there are cases where it cannot behave as a self-contained Swift toolchain. For example, there can be external dependencies on specific files, such as headers or libraries, far too many and variable between toolchain versions to be managed by swiftly. Also, for long-lived processes, there is no way to gracefully restart them without help from the client.


		`swiftly install`

		If no swift version file can be found then the installation fails indicating that it couldn't fine the file.


		`swiftly use main-snapshot`

		The use subcommand also supports `.swift-version` files. If version file is present in the current working directory, or an ancestory directory, then swiftly will update that file with the new version to use. This can be a useful feature for a team to share and align on toolchain versions with git. As a special case, if swiftly could not find a version file, but it could find a Package.swift file it will create a new version file for you in the package and set that to the requested toolchain version.

	Install whatever toolchain is current selected, such as a .swift-version file:
	Install whatever toolchain is currently selected, such as a .swift-version file:

		Run a command while proxying to the selected toolchain commands.


		Run a command with a selected toolchain, so that all toolchain commands are become the default added to the system path and other common environment variables.

		The `~/.local/bin` directory would include symlinks pointing to the `bin` directory of the "active" toolchain, if any.
		The `~/.local/bin` directory would include symlinks pointing to swiftly itself. When the proxies binaries are executed swiftly proxies them to the requested toolchain, or the default.

Swiftly proxies #155

Swiftly proxies #155

Uh oh!

Conversation

cmcgee1024 commented Aug 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BrianHenryIE commented Aug 18, 2024

Uh oh!

cmcgee1024 commented Aug 20, 2024

Uh oh!

adam-fowler left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cmcgee1024 commented Aug 20, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

0xTim left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cmcgee1024 commented Aug 26, 2024

Uh oh!

patrickfreed left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

0xTim commented Aug 27, 2024

Uh oh!

airspeedswift commented Aug 27, 2024

Uh oh!

cmcgee1024 commented Aug 28, 2024

Uh oh!

cmcgee1024 commented Sep 4, 2024

Uh oh!

rauhul commented Sep 5, 2024

Uh oh!

cmcgee1024 commented Sep 5, 2024

Uh oh!

cmcgee1024 commented Sep 6, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cmcgee1024 commented Aug 10, 2024 •

edited

Loading

patrickfreed left a comment •

edited

Loading

al45tair Nov 8, 2024 •

edited

Loading

al45tair Nov 8, 2024 •

edited

Loading