Add an "invisible" flag to require packages to be installed explicitly

[kit-ty-kate]

This is a follow up of #4432

I think avoid-version should stay as-is (it's too late to rename the variable anyway IMO) but I'd like to propose a new flag: invisible which would make the package flagged with it effectively unavailable unless already installed or explicitly required (either via a lock file or an explicit call to opam install).

This would help opam-repository greatly as currently the way to """remove""" packages (i.e. available: false) effectively breaks any body using it explicitly in CI systems, lock files or that already installed it.

Would help partially fix ocaml/opam-repository#10531

[dra27]

I don't think it's too late to rename hidden-version - it's literally used by the compiler's beta packages only, and we haven't flipped opam 2.1 to rc. But I'll open an issue to remind us (I should have done that around the original discussion )

[dra27]

Update from developer meeting: this should be spun into a full spec. It may also be possible to implement as a variable (to be used in the available field) rather than as a flag, while maintaining compatibility with opam 2.0 and 2.1.

The goal - which cannot be achieved with avoid-version - is that packages can be disabled in opam-repository without breaking existing setups and lock files, but we need to flesh out the specification to make sure everything's covered (dependencies on invisible packages, opam show, etc.).

It's definitely orthogonal to the avoid-version flag used for the compilers, where being able to explicitly install a package which depends on a pre-released package is the required feature.

[dra27]

(previous comments updated just to reflect that we did rename hidden-version to avoid-version, but the need for this issue remains!)

[dra27]

Thoughts from dev meeting to do this in 2.2: available: if-requested would have the benefit of being silently treated as available: false in opam 2.0 and 2.1 (no warning for an unknown flag, etc.) and @AltGr estimates the implementation to be a little simpler that way. There's a benefit for error messages if only the invisible package would work, since opam already displays errors for no solutions owing to unavailable packages - they could be tweaked to recognise that the package is marked if-required (or whatever it's called).

[AltGr]

I had a quick look on what @dra27 just described, and unfortunately it's not as simple as I first thought: variables like with-doc which rely on the package being explicitely selected are only available in the depends: and similar fields, and the availability is actually computed way before that, at switch state load, when there is no notion of a current request.

Ah also, implementation wise it would be if-requested-or-installed of course.

[kit-ty-kate]

We could implement this by adding a new field requested in switch_state, requiring an (optional?) list of requested packages for each time we create a new state. It would be empty by default which would make all the packages invisible and only when given explicitly through OpamClient.install or similar functions.

Or we could change switch_state to be a phantom type of arity 2, first being in a state where availability is not decided yet, and unlock availability when OpamAuxCommands.autopin is called.