Skip to content

Commit 682c54a

Browse files
mergify[bot]mergify[bot]
authored
Merge pull request #10731 from mpickering/wip/t10692
cabal-install: Be less eager to configure external programs
2 parents bf432d2 + 2c19bf3 commit 682c54a

File tree

22 files changed

+200
-8
lines changed

22 files changed

+200
-8
lines changed

‎cabal-install/src/Distribution/Client/ProjectPlanning.hs

Lines changed: 45 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -484,6 +484,7 @@ configureCompiler
484484
let fileMonitorCompiler = newFileMonitor $ distProjectCacheFile "compiler"
485485

486486
progsearchpath <- liftIO $ getSystemSearchPath
487+
487488
rerunIfChanged
488489
verbosity
489490
fileMonitorCompiler
@@ -499,7 +500,7 @@ configureCompiler
499500
let extraPath = fromNubList packageConfigProgramPathExtra
500501
progdb <- liftIO $ prependProgramSearchPath verbosity extraPath [] defaultProgramDb
501502
let progdb' = userSpecifyPaths (Map.toList (getMapLast packageConfigProgramPaths)) progdb
502-
(comp, plat, progdb'') <-
503+
result@(_, _, progdb'') <-
503504
liftIO $
504505
Cabal.configCompilerEx
505506
hcFlavor
@@ -516,17 +517,55 @@ configureCompiler
516517
-- programs it cares about, and those are the ones we monitor here.
517518
monitorFiles (programsMonitorFiles progdb'')
518519

519-
-- Configure the unconfigured programs in the program database,
520-
-- as we can't serialise unconfigured programs.
521-
-- See also #2241 and #9840.
522-
finalProgDb <- liftIO $ configureAllKnownPrograms verbosity progdb''
520+
-- Note: There is currently a bug here: we are dropping unconfigured
521+
-- programs from the 'ProgramDb' when we re-use the cache created by
522+
-- 'rerunIfChanged'.
523+
--
524+
-- See Note [Caching the result of configuring the compiler]
523525

524-
return (comp, plat, finalProgDb)
526+
return result
525527
where
526528
hcFlavor = flagToMaybe projectConfigHcFlavor
527529
hcPath = flagToMaybe projectConfigHcPath
528530
hcPkg = flagToMaybe projectConfigHcPkg
529531

532+
{- Note [Caching the result of configuring the compiler]
533+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
534+
We can't straightforwardly cache anything that contains a 'ProgramDb', because
535+
the 'Binary' instance for 'ProgramDb' discards all unconfigured programs.
536+
See that instance, as well as 'restoreProgramDb', for a few more details.
537+
538+
This means that if we try to cache the result of configuring the compiler (which
539+
contains a 'ProgramDb'):
540+
541+
- On the first run, we will obtain a 'ProgramDb' which may contain several
542+
unconfigured programs. In particular, configuring GHC will add tools such
543+
as `ar` and `ld` as unconfigured programs to the 'ProgramDb', with custom
544+
logic for finding their location based on the location of the GHC binary.
545+
- On subsequent runs, if we use the cache created by 'rerunIfChanged', we will
546+
deserialise the 'ProgramDb' from disk, which means it won't include any
547+
unconfigured programs, which might mean we are unable to find 'ar' or 'ld'.
548+
549+
This is not currently a huge problem because, in the Cabal library, we eagerly
550+
re-run the configureCompiler step (thus recovering any lost information), but
551+
this is wasted work that we should stop doing in Cabal, given that cabal-install
552+
has already figured out all the necessary information about the compiler.
553+
554+
To fix this bug, we can't simply eagerly configure all unconfigured programs,
555+
as was originally attempted, for a couple of reasons:
556+
557+
- it does more work than necessary, by configuring programs that we may not
558+
end up needing,
559+
- it means that we prioritise system executables for built-in build tools
560+
(such as `alex` and `happy`), instead of using the proper version for a
561+
package or package component, as specified by a `build-tool-depends` stanza
562+
or by package-level `extra-prog-path` arguments.
563+
This lead to bug reports #10633 and #10692.
564+
565+
See #9840 for more information about the problems surrounding the lossly
566+
Binary ProgramDb instance.
567+
-}
568+
530569
------------------------------------------------------------------------------
531570

532571
-- * Deciding what to do: making an 'ElaboratedInstallPlan'

‎cabal-testsuite/PackageTests/BuildToolDependsExternal/cabal.project

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
packages: client
2+
optional-packages: pre-proc

‎cabal-testsuite/PackageTests/BuildToolDependsExternal/client/Hello.x

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
module Main where
2+
3+
main = print 0

‎cabal-testsuite/PackageTests/BuildToolDependsExternal/client/client.cabal

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
name: client
2+
version: 0.1.0.0
3+
synopsis: Checks build-tool-depends are put in PATH
4+
license: BSD3
5+
category: Testing
6+
build-type: Simple
7+
cabal-version: >=1.10
8+
9+
executable hello-world
10+
main-is: Hello.hs
11+
build-depends: base
12+
build-tool-depends: pre-proc:alex
13+
default-language: Haskell2010

‎cabal-testsuite/PackageTests/BuildToolDependsExternal/pre-proc/MyCustomPreprocessor.hs

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
module Main where
2+
3+
import System.Environment
4+
import System.IO
5+
6+
-- This is a "fake" version of alex, so it should take the command line arguments
7+
-- as alex.
8+
main :: IO ()
9+
main = do
10+
(_:"-o":target:source:_) <- getArgs
11+
let f '0' = '1'
12+
f c = c
13+
writeFile target . map f =<< readFile source

‎cabal-testsuite/PackageTests/BuildToolDependsExternal/pre-proc/pre-proc.cabal

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
name: pre-proc
2+
version: 999.999.999
3+
synopsis: Checks build-tool-depends are put in PATH
4+
license: BSD3
5+
category: Testing
6+
build-type: Simple
7+
cabal-version: >=1.10
8+
9+
executable alex
10+
main-is: MyCustomPreprocessor.hs
11+
build-depends: base, directory
12+
default-language: Haskell2010
13+
14+
executable bad-do-not-build-me
15+
main-is: MyMissingPreprocessor.hs
16+
build-depends: base, directory
17+
default-language: Haskell2010

‎cabal-testsuite/PackageTests/BuildToolDependsExternal/scripts/alex

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
#! /usr/bin/env bash
2+
3+
echo "I am not the alex you are looking for"
4+
exit 1

‎cabal-testsuite/PackageTests/BuildToolDependsExternal/setup.out

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# cabal v2-build
2+
Configuration is affected by the following files:
3+
- cabal.project
4+
Resolving dependencies...
5+
Build profile: -w ghc-<GHCVER> -O1
6+
In order, the following will be built:
7+
- pre-proc-999.999.999 (exe:alex) (first run)
8+
- client-0.1.0.0 (exe:hello-world) (first run)
9+
Configuring executable 'alex' for pre-proc-999.999.999...
10+
Preprocessing executable 'alex' for pre-proc-999.999.999...
11+
Building executable 'alex' for pre-proc-999.999.999...
12+
Configuring executable 'hello-world' for client-0.1.0.0...
13+
Preprocessing executable 'hello-world' for client-0.1.0.0...
14+
Building executable 'hello-world' for client-0.1.0.0...

‎cabal-testsuite/PackageTests/BuildToolDependsExternal/setup.test.hs

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
import Test.Cabal.Prelude
2+
-- Test build-tool-depends isn't influenced by PATH
3+
main = cabalTest $ do
4+
env <- getTestEnv
5+
addToPath (testTmpDir env </> "scripts/") $ cabal "v2-build" ["client"]

‎cabal-testsuite/PackageTests/ExtraProgPath/setup.out

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
11
# cabal v2-build
2-
Warning: cannot determine version of <ROOT>/pkg-config :
3-
""
42
Configuration is affected by the following files:
53
- cabal.project
4+
Warning: cannot determine version of <ROOT>/pkg-config :
5+
""
6+
Warning: cannot determine version of <ROOT>/pkg-config :
7+
""
68
Resolving dependencies...
79
Error: [Cabal-7107]
810
Could not resolve dependencies:

0 commit comments

Comments
 (0)