Use Gleam version as a constraint in package resolution

[mine-tech-oficial]

As the title describes, allow package resolution to consider the gleam field in gleam.toml as a constraint check, so that when packages upgrade to a newer Gleam version, users of it aren't required to manually adjust its range

[giacomocavalieri]

Would this be a net positive? Wouldn't we want to encourage folks to always keep their gleam version up to date?

[mine-tech-oficial]

I would say that, if for some reason someone needs to support a specific version, they don't need to worry about bugs each time they update. An example (which is what, I suppose, quite some people stumbled on) was the (unofficial) Gleam Chess Tournament, as the code ran on container with Gleam v1.9.1.

[silby]

I would also have benefitted from this today. OpenBSD 7.7 has Gleam 1.9.1 in ports and OpenBSD ports/packages are only backported to the stable release in a somewhat ad hoc way/when there are security issues justifying the effort. I ran a package bump with Gleam 1.11.0 on my desktop but then had a compatibility issue with 1.9.1 on my server. I was able to tweak my manifest to fix it by constraining the package at issue.

[lpil]

@silby You must never edit the manifest. You should edit gleam.toml.

[silby]

Oh I see I was misleading. I induced a change in the manifest by editing gleam.toml. Clear communication eludes me this week.

[lpil]

Ah right! Thanks

[mtlynch]

This just bit me as well. I'm on NixOS, where the latest version of Gleam available is Gleam 1.10.0 even though the latest version is Gleam 1.11.0.

If I create a new project with gleam new, it pulls in a version of gleeunit that's incompatible with my version of Gleam:

$ gleam --version
gleam 1.10.0

$ gleam new --name myproject --skip-github .
Your Gleam project myproject has been successfully created.
The project can be compiled and tested by running these commands:

        gleam test
$ gleam run
  Resolving versions
Downloading packages
 Downloaded 2 packages in 0.01s
  Compiling gleam_stdlib
error: Incompatible Gleam version

The package `gleeunit` requires a Gleam version satisfying 1.11.0 <= v but you are using v1.10.0.

gleeunit 1.4.0 is correctly declaring its dependency on Gleam 1.11.0, so it would be helpful if gleam new considered that when choosing which version of gleeunit to pull in for a new project.

[manveru]

I was bit by the same issue, luckily NixOS/nixpkgs#407394 bumped Gleam to 1.11.0, but I think generally we should make use of the information that package maintainers provide to resolve versions properly.

[silby]

Herein I harvest and expand on a comment in closed #5008 into this open discussion instead, where it's more appropriate.

In brief: the gleam version lower bound constraint in gleam.toml should be applied to dependency resolution when constructing a manifest, such that every dependency in the manifest declares a lesser or equal lower bound to the lower bound of the current package.

Rationale

I take it as a given that it is good, not bad, for package repositories to package and distribute Gleam, as some in fact do. Projects have different approaches to release stability, ranging from rolling-release (e.g. Arch Linux, Void Linux, Homebrew) to medium-term stabilized releases cut from a single head (e.g. OpenBSD -release versions) to long-term branched and stabilized releases (Debian stable). The simplest way (and in certain cases only practical way) for many users to install Gleam will be through OS packages.

This ultimately leads to the phenomenon, well-known from Debian stable users, of users being stuck on out-of-date versions of things, to the periodic chagrin of upstreams who get bug reports that should be going to Debian package maintainers. In Gleam's case, Debian release engineering has posed an obstacle to packaging Gleam before it even gets particularly close, cf. #3160, but we can suppose that if anyone ever goes for it there will be Debian users getting old versions of Gleam when they apt install gleam.

OpenBSD releases, as discussed in #5008, are always fresher than Debian stable, but whatever Gleam version is built from the ports tree in the spring and fall will generally be the only one packaged for the release version outside of a security issue. OpenBSD also doesn't guarantee forward- or backward-compatibility for binaries you build on it, so my Gleam project hosted on my OpenBSD VPS pretty much has to be built there, using the packaged version of Gleam and the system toolchain and libc for native dependencies. I'm not incapable of building a newer version of the Gleam port for myself on a release version of OpenBSD but it's a yak I choose not to shave. The alternative I have is basically trial-and-error pinning of my dependencies, including transitive ones. My gleam.toml now contains lines like:

gleam_stdlib = "0.60.0"
exception = "2.0.0"
gleam_time = "< 1.3.0"

all of which I had to go inspect source history of the packages to get right.

Even users that use an up-to-date Gleam on their own machines, perhaps packaged by upstream, may have to build and deploy code on systems that are standardized on an older Gleam. Even a lag of a few working days in upgrading a shared build system from one Gleam minor version to the next can cause a bad developer experience, when, for example, a SemVer-safe version bump to the stdlib done by a bot causes a build failure.

What do others do?

Obviously in the case of Ruby and Python, unlike Gleam, there are support policies for the multiple annual releases of the interpreter, so the demand/use-case for this feature is more mature. Still, two examples:

uv

With a pyproject.toml containing:

requires-python = ">=3.10, <3.11"
dependencies = ["numpy"]

uv lock resolves to numpy 2.2.6. numpy 2.3.0 doesn't support Python 3.10.

bundler

With a gemfile containing:

ruby "~> 3.1"
gem "plissken"

bundle install resolves to plissken 3.0.0. plissken 4.0.0 doesn't support Ruby 3.1.

What's the impact of doing nothing?

I don't know much about your approach to evolving Gleam-the-language, gleam-the-compiler, or the stdlib. It's possible that after the stdlib hits 1.0 it'll become drastically less likely for old versions of Gleam 1.x to be unable to compile new code. But since you're explicitly not engineering for forward-compatibility (which is fine) I don't think you could easily make even a best-effort commitment here. Still, this might become less relevant soon. Without this feature though, in cases where the minimum Gleam version of dependencies advances within a major version, people in my situation have to laboriously pin package versions to avoid getting hosed by a gleam update.

[ToddG]

I'll just add that while I've switched from OpenBSD to FreeBSD for my production environment, I'm hitting the same issue in FreeBSD.

I've emailed the FreeBSD maintainer (dch@FreeBSD.org) to see what my options are on that side of the fence. Hopefully they can point me at a repo, patch, or instructions for updating gleam to the latest version.

REPRO

[[root@fbsdhost /jail]] # iocage console foo
Last login: Tue Jan 27 14:26:39 on pts/2
FreeBSD 15.0-RELEASE (GENERIC) releng/15.0-n280995-7aedc8de6446

root@foo:~ # which gleam
/usr/local/bin/gleam
root@foo:~ # gleam --version
gleam 1.11.0
root@foo:~ # mkdir projects
root@foo:~ # cd projects/
root@foo:~/projects # gleam new myapp
Your Gleam project myapp has been successfully created.
The project can be compiled and tested by running these commands:

	cd myapp
	gleam test

root@foo:~/projects # cd myapp
root@foo:~/projects/myapp # gleam test
  Resolving versions
Downloading packages
 Downloaded 2 packages in 0.25s
error: Incompatible Gleam version

The package `gleam_stdlib` requires a Gleam version satisfying 1.14.0 <= v but you are using v1.11.0.

[daniellionel01]

Hi!

Ran into this myself the other day. I think instead of using the gleam version as a constraint, we could do something more helpful and less complicated first: tell the user the latest compatible version.

I was blocked by this because I was testing for a regression on a previous gleam version and I would have appreciated if I got an error message similar to this:

# gleam test
  Resolving versions
Downloading packages
 Downloaded 2 packages in 0.25s
error: Incompatible Gleam version

The package `gleam_stdlib` requires a Gleam version satisfying 1.14.0 <= v but you are using v1.11.0.
Install `gleam_stdlib@0.67.0` for a compatible version.

Obviously the wording would have to improve. But you get the gist. The most "annoying" part was having to go back in the gleam.toml to verify when the gleam constraint was updated, since there is (to my knowledge) no faster way to check this.

Let me know what y'all think!

[lpil]

How would we determine the latest compatible version?

[lpil]

How would this be implemented?

It would need to work with Hex registries and not just path dependencies.

[daniellionel01]

I can think of a couple of ways a tool could do this, but nothing that is 100% reliable, fast, or integrated into the hex registry.

With hex you could download the tarballs, unpack them, parse the gleam.toml and stop when you find a compatible version.
Or you lookup the repository from the hex api and check for the release commits and lookup the gleam.toml through the github api.
But that also wouldn't guarantee the version constraints with any other package you might have installed.

So yeah doesn't seem worth the effort right now.
I might hack together a bash script that does this, since I would just use it for checking for regressions.

[lpil]

Downloading the tarballs would be extremely slow, and the API isn't available for all repos. :(

[daniellionel01]

For hex packages you could make use of https://preview.hex.pm/preview/gleam_stdlib/0.69.0/show/gleam.toml hehe and parse the html (not a serious suggestion).

Quite ambitious, but https://packages.gleam.run/ (or a separate API) could keep its own database of the full gleam.toml and even index on the gleam constraint of all packages published through Gleam. I'm sure there are going to be future use cases where more package information than what hex gives us is useful.

[lpil]

That's again quite expensive, and becoming a load bearing infrastructure provider is a huge amount of work.