Standard Library Stability

[Sgregory42]

(note I used an llm for formatting, the thoughts are my own)

Standard Library Stability Discussion

I want to open a discussion about a backward incompatible change in Gleam's stdlib and why IMHO it shouldn't happen again.

What Happened

From my understanding, some of the dynamic/decode library were deprecated and then removed in the same minor versions (0.59 to 0.61).

The Issue

I agree that a library before a v1 release is prone to API changes, but a standard library is not just a random library. It's the library that everyone depends on. With Gleam being v1, there is an expectation of stability with the standard library.

I want to raise awareness that the stdlib should have the same care as Gleam itself in terms of stability and consideration when making changes.

Impact

The Gleam libraries ecosystem is small and this change just fractured it even more.
As a new dev in Gleam, as I finally reach an acceptable level of productivity with just the language, it gets shattered whenever I want to use a package and I have to deal with dependency issues. (Maybe I am the only one having these issues 🤷 )

In a business environment where dev time is of the essence, it is very hard to justify time lost on this kind of issues especially in a
new language. The goal is to show how productive we are and how this amazing language solves business problems.
Solving business problems means more adoption, which means contributing back to the language and supporting it.

Possible solution

From my basic understanding, erlang allows to only have one module for the app, hence each library can't have their own version of the stdlib. (correct me if I am wrong)
Obviously easiest is to not make any non backward compatible change until V1, but if there is a need to:

• In C++, one solution is to use namespaces with version 1 or 2 when we need to have two version at the same time, not sure if it is something possible here.
• Or maybe the breaking changes can be in a separate module, with deprecation warning to use the new module. When V1 hits (or any other major version change), it's just a matter of search and replace module name. Active maintainers can early adopt while others can do it at a slower pace until the major version change.

(Those are not ideal but kinda works until V1)

Any other way to prevent breaking changes within a same minor version ?

[lpil]

All breaking changes get a deprecation cycle, during which warnings are emitted for their use. Several months are given during which to fix any issues.

The two proposals you've given would mean an immediate breaking changes and no migration period. This would make the problem much worse.

Deprecation cycles will continue until the stdlib stabilises. This will likely be in the autumn.

[lpil]

I think you're looking at the wrong part of the C++ approach. The reason they don't feel the same pain is that their deprecation cycle is over many years or even decades, while ours is over months. This time scale makes sense for a library as old as these C++ ones, but it does not make sense for a version zero library for a language that's only just a year old.

If we were to adopt the same strategy we could expect the stdlib to stabilise some time after 2030 instead of in a few months. This isn't acceptable for us.

Moving to a new namespace would mean that all Gleam code ever written would need to be rewritten to use the new namespace, not just code that uses the old API. This isn't acceptable either.

Either way it's several years too late to propose changes to how this is done. We're at the end of v0 so even if we did find a different strategy that we like we would have no opportunity to use it.

[Sgregory42]

Even though the C++ version/stdlib evolve at a different pace than gleam, it is still the users choice to update or not and with the expectation of not breaking contracts in minor versions.

I think the main problem here is the gleam stdlib is still at V0 with no proper mechanism to introduce breaking changes that adds an maintenance cost.
Since it seems to "soon" reach V1, there is not much to be done as you mentioned.

Could you confirm that post V1 all minor version will be backward compatible ?

[lpil]

it is still the users choice to update or not and with the expectation of not breaking contracts in minor versions.

That would mean the following:

• We maintain tens of incompatible versions of the standard library concurrently, fracturing the ecosystem into the same number of incompatible parts depending on what "version" of the stdlib they are using.
• We are unable to force people to migrate to the latest version, so the ecosystem would drift away from the canonical version resulting in:
  • A proliferation of code and patterns that we have deliberately made difficult or impossible today.
  • No libraries being ready for the migration to v1, possibly resulting in general failure to migrate due to the size of the work being too large. A Python-3 style problem.

This would culminate in a significantly more painful version of the problem you are describing at the start.

Could you confirm that post V1 all minor version will be backward compatible ?

Yes, that is what v1 means in semver, which the Gleam ecosystem uses

[Sgregory42]

Thank you for your insight, it does make some sense.
I think for now, it's just a matter of waiting for the V1 for the stability I need as a user but also as a potential package contributor.
We can probably close this thread :)

[lpil]

You don't need to wait for v1, the deprecation cycles have ended, as I said.