Skip to content

Commit 6acff88

Browse files
committed
Tidy up lock files
1 parent 7058221 commit 6acff88

File tree

1 file changed

+81
-86
lines changed

1 file changed

+81
-86
lines changed

doc/lock_files.md

Lines changed: 81 additions & 86 deletions
Original file line numberDiff line numberDiff line change
@@ -2,123 +2,113 @@
22

33
# Lock Files
44

5-
Stack attempts to provide reproducible build plans. This involves
6-
reproducibly getting the exact same contents of source packages and
7-
configuration options (like cabal flags and GHC options) for a given
8-
set of input files. There are a few problems with making this work:
9-
10-
* Entering all of the information to fully provide reproducibility is
11-
tedious. This would include things like Hackage revisions, hashes of
12-
remote tarballs, etc. Users don't want to enter this information.
13-
* Many operations in Stack rely upon a "snapshot hash," which
14-
transitively includes the completed information for all of these
15-
dependencies. If any of that information is missing when parsing the
16-
`stack.yaml` file or snapshot files, it could be expensive for Stack
17-
to calculate it.
5+
Stack attempts to provide reproducible build plans. This involves reproducibly
6+
getting the exact same contents of source packages and configuration options
7+
(like Cabal flags and GHC options) for a given set of input files. There are a
8+
few problems with making this work:
9+
10+
* Entering all of the information to fully provide reproducibility is tedious.
11+
This would include things like Hackage revisions, hashes of remote tarballs,
12+
etc. Users don't want to enter this information.
13+
* Many operations in Stack rely upon a "snapshot hash," which transitively
14+
includes the completed information for all of these dependencies. If any of
15+
that information is missing when parsing the `stack.yaml` file or snapshot
16+
files, it could be expensive for Stack to calculate it.
1817

1918
To address this, we follow the (fairly standard) approach of having a
20-
_lock file_. The goal of the lock file is to cache completed
21-
locations of project, snapshot packages and snapshots themselves so that:
19+
_lock file_. The goal of the lock file is to cache completed locations of
20+
project, snapshot packages and snapshots themselves so that:
2221

2322
* These files can be stored in source control
24-
* Users on other machines can reuse these lock files and get identical
25-
build plans given that the used local packages and local snapshots are
26-
the same on those machines
27-
* Rerunning `stack build` in the future is deterministic in the build
28-
plan, not depending on mutable state in the world like Hackage
29-
revisions
30-
* **NOTE** If, for example, a tarball available remotely is
31-
deleted or the hash changes, it will not be possible for Stack
32-
to perform the build. However, by deterministic, we mean it
33-
either performs the same build or fails, never accidentally
34-
doing something different.
35-
36-
This document explains the contents of a lock file, how they are used,
37-
and how they are created and updated.
23+
* Users on other machines can reuse these lock files and get identical build
24+
plans given that the used local packages and local snapshots are the same on
25+
those machines
26+
* Rerunning `stack build` in the future is deterministic in the build plan, not
27+
depending on mutable state in the world like Hackage revisions
3828

39-
## stack.yaml and snapshot files
29+
!!! note
30+
31+
If, for example, a tarball available remotely is deleted or the hash
32+
changes, it will not be possible for Stack to perform the build. However,
33+
by deterministic, we mean it either performs the same build or fails,
34+
never accidentally doing something different.
4035

41-
Relevant to this discussion, the `stack.yaml` file specifies:
36+
This document explains the contents of a lock file, how they are used, and how
37+
they are created and updated.
4238

43-
* Resolver (the parent snapshot)
44-
* `extra-deps`
39+
## stack.yaml and snapshot files
4540

46-
The resolver can either specify a compiler version or another snapshot
47-
file. This snapshot file can contain the same information referenced
48-
above for a `stack.yaml`, with the following differences:
41+
Relevant to this discussion, Stack's project-level configuration file
42+
(`stack.yaml`) specifies:
4943

50-
* The `extra-deps` are called `packages`
51-
* Drop packages can be included
44+
* the parent snapshot (`resolver` or `snapshot`)
45+
* extra-deps
5246

53-
Some information in these files can be incomplete. Consider:
47+
Some of this information can be incomplete. Consider this `stack.yaml` file:
5448

5549
~~~yaml
56-
resolver: lts-13.9
57-
packages: []
50+
resolver: lts-19.22
51+
packages:
52+
- .
5853
extra-deps:
59-
- https://hackage.haskell.org/package/acme-missiles-0.3.tar.gz
54+
- acme-missiles-0.3
6055
~~~
6156

62-
This information is _incomplete_, since the contents of that URL may
63-
change in the future. Instead, you could specify enough information in
64-
the `stack.yaml` file to fully resolve that package. That looks like:
57+
This information is _incomplete_. For example, the extra-deps may change in the
58+
future. Instead, you could specify enough information in the `stack.yaml` file
59+
to fully resolve that package. That looks like:
6560

6661
~~~yaml
6762
extra-deps:
68-
- size: 1442
69-
url: https://hackage.haskell.org/package/acme-missiles-0.3.tar.gz
70-
name: acme-missiles
71-
version: '0.3'
72-
sha256: e563d8b524017a06b32768c4db8eff1f822f3fb22a90320b7e414402647b735b
63+
- hackage: acme-missiles-0.3@sha256:2ba66a092a32593880a87fb00f3213762d7bca65a687d45965778deb8694c5d1,613
7364
pantry-tree:
7465
size: 226
7566
sha256: 614bc0cca76937507ea0a5ccc17a504c997ce458d7f2f9e43b15a10c8eaeb033
7667
~~~
7768

78-
Users don't particularly feel like writing all of that. Therefore,
79-
it's common to see _incomplete_ information in a `stack.yaml` file.
80-
81-
Additionally, the `lts-13.9` information is _also_ incomplete. While
82-
we assume in general that LTS snapshots never change, there's nothing
83-
that technically prohibits that from happening. Instead, the complete
84-
version of that field is:
69+
The `lts-19.22` information is also incomplete. While we assume in general that
70+
Haskell LTS snapshots never change, there's nothing that prohibits that from
71+
happening. Instead, the complete version of that key is:
8572

8673
~~~yaml
8774
resolver:
88-
size: 496662
89-
url: https://raw.githubusercontent.com/commercialhaskell/stackage-snapshots/master/lts/13/9.yaml
90-
sha256: 83de9017d911cf7795f19353dba4d04bd24cd40622b7567ff61fc3f7223aa3ea
75+
- url: https://raw.githubusercontent.com/commercialhaskell/stackage-snapshots/master/lts/19/22.yaml
76+
size: 619399
77+
sha256: 5098594e71bdefe0c13e9e6236f12e3414ef91a2b89b029fd30e8fc8087f3a07
9178
~~~
9279

93-
Also something people don't feel like writing by hand.
80+
Users don't particularly feel like writing all of that. Therefore, it's common
81+
to see _incomplete_ information in a `stack.yaml` file.
9482

9583
## Recursive snapshot layers
9684

97-
Snapshot files can be _recursive_, where `stack.yaml` refers to
98-
`foo.yaml`, which refers to `bar.yaml`, which refers to `baz.yaml`. A
99-
local snapshot file can refer to a remote snapshot file (available via
100-
an HTTP(S) URL).
85+
Snapshot files can be _recursive_, where `stack.yaml` refers to `foo.yaml`,
86+
which refers to `bar.yaml`, which refers to `baz.yaml`. A local snapshot file
87+
can refer to a remote snapshot file (available via an HTTP(S) URL).
10188

102-
We need to encode information from _all_ of these snapshot layers and
103-
the `stack.yaml` file in the lock file, to ensure that we can detect
104-
if anything changes.
89+
We need to encode information from _all_ of these snapshot layers and the
90+
`stack.yaml` file in the lock file, to ensure that we can detect if anything
91+
changes.
10592

10693
## Performance
10794

108-
In addition to acting as a pure correctness mechanism, the design of a
109-
lock file given here also works as a performance improvement. Instead
110-
of requiring that all snapshot files be fully parsed on each Stack
111-
invocation, we can store information in the lock file and bypass
112-
parsing of the additional files in the common case of no changes.
95+
In addition to acting as a pure correctness mechanism, the design of a lock file
96+
given here also works as a performance improvement. Instead of requiring that
97+
all snapshot files be fully parsed on each Stack invocation, we can store
98+
information in the lock file and bypass parsing of the additional files in the
99+
common case of no changes.
113100

114101
## Lock file contents
115102

116103
The lock file contains the following information:
117104

118-
* Completed package locations for both `extra-deps` and packages in
119-
snapshot files
120-
* **NOTE** This only applies to _immutable_ packages. Mutable
121-
packages are not included in the lock file.
105+
* Completed package locations for extra deps and packages in snapshot files
106+
107+
!!! note
108+
109+
This only applies to _immutable_ packages. Mutable packages are not
110+
included in the lock file.
111+
122112
* Completed information for the snapshot locations
123113

124114
It looks like the following:
@@ -157,11 +147,16 @@ packages:
157147

158148
## Creation
159149

160-
Whenever a `stack.yaml` file is loaded, Stack checks for a lock file
161-
in the same file path, with a `.lock` extension added. For example, if
162-
you run `stack build --stack-yaml stack-11.yaml`, it will use a lock
163-
file in the location `stack-11.yaml.lock`. For the rest of this
164-
document, we'll assume that the files are simply `stack.yaml` and
150+
Whenever a project-level configuration file (`stack.yaml`) is loaded, Stack
151+
checks for a lock file in the same file path, with a `.lock` extension added.
152+
For example, if you command:
153+
154+
~~~text
155+
stack build --stack-yaml my-stack.yaml
156+
~~~
157+
158+
Stack will use a lock file in the location `my-stack.yaml.lock`. For the rest of
159+
this document, we'll assume that the files are simply `stack.yaml` and
165160
`stack.yaml.lock`.
166161

167162
If the lock file does not exist, it will be created by:
@@ -173,7 +168,7 @@ If the lock file does not exist, it will be created by:
173168

174169
## Update procedure
175170

176-
When loading a Stack project all completed package or snapshot locations
177-
(even when they were completed using information from a lock file) get
178-
collected to form a new lock file in memory and compare against the one
179-
on disk, writing if there are any differences.
171+
When loading a Stack project all completed package or snapshot locations (even
172+
when they were completed using information from a lock file) get collected to
173+
form a new lock file in memory and compare against the one on disk, writing if
174+
there are any differences.

0 commit comments

Comments
 (0)