Skip to content

Conversation

TeofilC
Copy link
Collaborator

@TeofilC TeofilC commented Jun 9, 2025

Summary

This change allows reinstalling packages like base and
template-haskell with GHC >=9.14, and it doesn't modify the behaviour
of cabal-install with older versions of GHC for backwards-compatibility.

Context

cabal-install includes a hardcoded list of non-reinstallable packages.
cabal-install will refuse to use a different version of these packages
in a build plan unless the --allow-boot-library-installs flag is set.

This list of packages is too pessimistic, and needlessly coupled to GHC.
For instance, as of GHC-9.12, base and template-haskell can be
"re-installed" without issue.

Change

This patch allows compilers to specify exactly which packages are wired-in
and so should not be reinstalled.

In the case of GHC-9.14+, this amounts to ghc-internal and ghc itself.

If a compiler chooses to specify this information then it overrides the
hardcoded non-reinstallable package list. Otherwise, it is still used
for the sake of backwards compatibility.

Note that this information comes in the form of unit-ids rather than
package names. This patch extends the solver with constraints that force
the use of a precise unit-id.

The behaviour here is still somewhat pessimistic. In the future, we
could further relax this restriction and only ban reinstalling these
packafes when absolutely necessary, eg, only ban re-installing ghc if
we are using plugins, etc.

For the GHC change that added the interface to expose the unit-id of
ghc-internal, see: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13297

Resolves #10087

Manual QA Notes

Build a compiler from this branch: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13297

and then confirm that when used with cabal from this branch it allows you to reinstall template-haskell for instance.

For instance this is what happens if I try to build template-haskell with a released version of cabal-install:

cabal output

❯ cabal build --with-ghc=$GHC template-haskell
Warning: Both /home/teo/.cabal and /home/teo/.config/cabal/config exist -
ignoring the former.
It is advisable to remove one of them. In that case, we will use the remaining
one by default (unless '$CABAL_DIR' is explicitly set).
Resolving dependencies...
Error: [Cabal-7107]
Could not resolve dependencies:
[__0] next goal: template-haskell (user goal)
[__0] rejecting: template-haskell-2.23.0.0 (constraint from non-reinstallable package requires installed instance)
[__0] rejecting: template-haskell; 2.22.0.0, 2.21.0.0, 2.20.0.0, 2.19.0.0, 2.18.0.0, 2.17.0.0, 2.16.0.0, 2.15.0.0, 2.14.0.0, 2.13.0.0, 2.12.0.0, 2.11.1.0, 2.11.0.0, 2.10.0.0, 2.9.0.0, 2.8.0.0, 2.7.0.0, 2.6.0.0, 2.5.0.0, 2.4.0.1, 2.4.0.0, 2.3.0.1, 2.3.0.0, 2.2.0.0 (constraint from user target requires ==2.23.0.0)
[__0] fail (backjumping, conflict set: template-haskell)
After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: template-haskell

versus from this branch

cabal output

Warning: this is a debug build of cabal-install with assertions enabled.
Warning: Both /home/teo/.cabal and /home/teo/.config/cabal/config exist -
ignoring the former.
It is advisable to remove one of them. In that case, we will use the remaining
one by default (unless '$CABAL_DIR' is explicitly set).
Warning: Parsing the index cache failed (Data.Binary.Get.runGet at position
16: Non-matching structured hashes: f46da61e7afa58a5e8fd1d2b6fb79899;
expected: d81bdd513f41b5d7ee4cd28455adadbe). Trying to regenerate the index
cache...
Resolving dependencies...
Build profile: -w ghc-9.13.20250617 -O1
In order, the following will be built (use -v for more details):
 - template-haskell-2.23.0.0 (lib) (first run)
Warning: this is a debug build of cabal-install with assertions enabled.
Configuring library for template-haskell-2.23.0.0...
Warning: this is a debug build of cabal-install with assertions enabled.
Preprocessing library for template-haskell-2.23.0.0...
Building library for template-haskell-2.23.0.0...
[ 1 of 11] Compiling Language.Haskell.TH.LanguageExtensions ( Language/Haskell/TH/LanguageExtensions.hs, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH/LanguageExtensions.o, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH/LanguageExtensions.dyn_o )
[ 2 of 11] Compiling Language.Haskell.TH.Ppr ( Language/Haskell/TH/Ppr.hs, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH/Ppr.o, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH/Ppr.dyn_o )
[ 3 of 11] Compiling Language.Haskell.TH.PprLib ( Language/Haskell/TH/PprLib.hs, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH/PprLib.o, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH/PprLib.dyn_o )
[ 4 of 11] Compiling Language.Haskell.TH.Quote ( Language/Haskell/TH/Quote.hs, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH/Quote.o, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH/Quote.dyn_o )
[ 5 of 11] Compiling System.FilePath.Posix ( vendored-filepath/System/FilePath/Posix.hs, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/System/FilePath/Posix.o, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/System/FilePath/Posix.dyn_o )
[ 6 of 11] Compiling System.FilePath  ( vendored-filepath/System/FilePath.hs, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/System/FilePath.o, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/System/FilePath.dyn_o )
[ 7 of 11] Compiling Language.Haskell.TH.Syntax ( Language/Haskell/TH/Syntax.hs, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH/Syntax.o, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH/Syntax.dyn_o )
[ 8 of 11] Compiling Language.Haskell.TH.Lib ( Language/Haskell/TH/Lib.hs, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH/Lib.o, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH/Lib.dyn_o )
[ 9 of 11] Compiling Language.Haskell.TH.CodeDo ( Language/Haskell/TH/CodeDo.hs, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH/CodeDo.o, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH/CodeDo.dyn_o )
[10 of 11] Compiling Language.Haskell.TH ( Language/Haskell/TH.hs, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH.o, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/Language/Haskell/TH.dyn_o )
[11 of 11] Compiling System.FilePath.Windows ( vendored-filepath/System/FilePath/Windows.hs, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/System/FilePath/Windows.o, dist-newstyle/build/x86_64-linux/ghc-9.13.20250617/template-haskell-2.23.0.0/build/System/FilePath/Windows.dyn_o )
Warning: this is a debug build of cabal-install with assertions enabled.


Include the following checklist in your PR:

@TeofilC TeofilC force-pushed the wip/teo-precise-unit-id branch 2 times, most recently from 6cfbe6a to e49a4b5 Compare June 19, 2025 17:45
@TeofilC TeofilC changed the title Wip/teo precise unit Add support for getting a list of non-reinstallable packages from the compiler Jun 19, 2025
@TeofilC TeofilC force-pushed the wip/teo-precise-unit-id branch 2 times, most recently from ecb3159 to dd63604 Compare June 19, 2025 17:51
@TeofilC TeofilC changed the title Add support for getting a list of non-reinstallable packages from the compiler Add Support getting a list of nonreinstallable units from GHC Jun 19, 2025
@TeofilC TeofilC changed the title Add Support getting a list of nonreinstallable units from GHC Add support getting a list of nonreinstallable units from GHC Jun 19, 2025
@TeofilC TeofilC force-pushed the wip/teo-precise-unit-id branch 2 times, most recently from 0bd8ca6 to 7b65354 Compare June 20, 2025 16:02
@TeofilC TeofilC marked this pull request as ready for review June 20, 2025 16:06
@TeofilC TeofilC changed the title Add support getting a list of nonreinstallable units from GHC Add support for getting a list of nonreinstallable units from GHC Jun 20, 2025
@TeofilC TeofilC force-pushed the wip/teo-precise-unit-id branch from 7b65354 to c035d6f Compare June 20, 2025 16:40
@TeofilC TeofilC changed the title Add support for getting a list of nonreinstallable units from GHC Support reinstalling more packages (including base and template-haskell) with GHC >9.14 Jun 20, 2025
@TeofilC TeofilC force-pushed the wip/teo-precise-unit-id branch 2 times, most recently from 3f1729d to f087826 Compare June 21, 2025 11:49
@Bodigrim
Copy link
Collaborator

Bodigrim commented Jul 5, 2025

How does it affect solver's error messages, when it complains about impossible base / template-haskell requirements? Is it better or worse?

@TeofilC
Copy link
Collaborator Author

TeofilC commented Jul 5, 2025

How does it affect solver's error messages, when it complains about impossible base / template-haskell requirements? Is it better or worse?

This change would only have an impact for GHC >9.14, base and template-haskell would be treated just like a normal package, so you'd get similar error messages to other boot libraries.

Do you have a particular solver error in mind?

@TeofilC TeofilC force-pushed the wip/teo-precise-unit-id branch 2 times, most recently from 730f5d5 to 99ea7c2 Compare July 7, 2025 12:40
@TeofilC TeofilC changed the title Support reinstalling more packages (including base and template-haskell) with GHC >9.14 Support reinstalling more packages (including base and template-haskell) with GHC >=9.14 Jul 7, 2025
Copy link
Collaborator

@sebright sebright left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, except that it needs tests. It would be great to have solver tests in https://github.com/haskell/cabal/blob/master/cabal-install/tests/UnitTests/Distribution/Solver/Modular/Solver.hs, including one that tests the new error message.

I wonder if we'll need to update the heuristics in the solver that assume that there is only one version of base (though not in this PR).

EDIT: I mainly reviewed the changes to the solver. I'm not familiar with the changes to GHC.

@ulysses4ever
Copy link
Collaborator

I was about to ask for solver tests as well.

@TeofilC
Copy link
Collaborator Author

TeofilC commented Aug 1, 2025

Thanks folks! I'll try to come back to this and add some tests soon (approx in the next few weeks)

@TeofilC TeofilC force-pushed the wip/teo-precise-unit-id branch 3 times, most recently from f54ff0f to 8842fdf Compare August 24, 2025 13:35
@TeofilC
Copy link
Collaborator Author

TeofilC commented Aug 24, 2025

I have added some test cases. I have extended the Solver options to allow setting wired in unit ids. As this setting is quite new, the reasonable default is to have it unset, and only set it for relevant test cases. In the future, this will flip and I will create a ticket, so we remember to update the default in a few years time.

@TeofilC TeofilC force-pushed the wip/teo-precise-unit-id branch 2 times, most recently from d520dd8 to 6fc61fb Compare August 24, 2025 13:50
In short, this change allows reinstalling packages like `base` and
`template-haskell` with GHC >=9.14, and it doesn't modify the behaviour
of cabal-install with older versions of GHC for backwards-compatibility.

cabal-install includes a hardcoded list of non-reinstallable packages.
cabal-install will refuse to use a different version of these packages
in a build plan unless the --allow-boot-library-installs flag is set.

This list of packages is too pessimistic, and needlessly coupled to GHC.
For instance, as of GHC-9.12, `base` and `template-haskell` can be
"re-installed" without issue.

This patch allows compilers to specify exactly which packages are wired-in
and so should not be reinstalled.

In the case of GHC >=9.14, this amounts to ghc-internal and ghc itself.

If a compiler chooses to specify this information then it overrides the
hardcoded non-reinstallable package list. Otherwise, it is still used
for the sake of backwards compatibility.

Note that this information comes in the form of unit-ids rather than
package names. This patch extends the solver with constraints that force
the use of a precise unit-id.

The behaviour here is still somewhat pessimistic. In the future, we
could further relax this restriction and only ban reinstalling these
packafes when absolutely necessary, eg, only ban re-installing `ghc` if
we are using plugins, etc.

For the GHC change that added the interface to expose the unit-id of
ghc-internal, see: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13297

Resolves #10087

Co-Authored-By: Matthew Pickering <[email protected]>
@TeofilC TeofilC force-pushed the wip/teo-precise-unit-id branch from 6fc61fb to f30b90c Compare August 24, 2025 14:11
@TeofilC TeofilC added the merge me Tell Mergify Bot to merge label Aug 29, 2025
@mergify mergify bot added ready and waiting Mergify is waiting out the cooldown period merge delay passed Applied (usually by Mergify) when PR approved and received no updates for 2 days labels Aug 29, 2025
mergify bot added a commit that referenced this pull request Aug 31, 2025
@mergify mergify bot merged commit ff61fb0 into master Aug 31, 2025
173 checks passed
@mergify mergify bot deleted the wip/teo-precise-unit-id branch August 31, 2025 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cabal-install: solver merge delay passed Applied (usually by Mergify) when PR approved and received no updates for 2 days merge me Tell Mergify Bot to merge ready and waiting Mergify is waiting out the cooldown period
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Amend non-reinstallable packages list
5 participants