Skip to content

Commit 172324e

Browse files
authored
Merge pull request cds-hooks#513 from buildpacks/fix-512-caching-strategies
Address issues in caching description
2 parents 5a82924 + 882774f commit 172324e

File tree

1 file changed

+8
-19
lines changed

1 file changed

+8
-19
lines changed
Lines changed: 8 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,42 +1,29 @@
11
+++
2-
title="Caching Strategies"
2+
title="Layer Types"
33
weight=6
44
summary="Learn strategies for caching layers."
55
+++
66

7-
# Caching
7+
# Layers
88

99
There are three types of layers that can be contributed to an image
1010

1111
* `build` layers -- the directory will be accessible by subsequent buildpacks,
1212
* `cache` layers -- the directory will be included in the cache,
1313
* `launch` layers -- the directory will be included in the run image as a single layer,
1414

15-
A fourth type of layer
16-
17-
* `ignored` layers
18-
19-
are available to buildpack authors for use as temporary layers.
20-
2115
In this section we look at caching each layer type.
2216

2317
## Layer Metadata
2418

25-
buildpacks ensure byte-for-byte reproducibility of layers. File creation time is [normalized to January 1, 1980](https://medium.com/buildpacks/time-travel-with-pack-e0efd8bf05db) to ensure reproducibility. Byte-for-byte reproducibility means previous layers can be reused. However, we want to invalidate previously cached layers if
26-
27-
* the buildpacks API changes,
28-
* the type of the layer changes.
29-
30-
A layer built using a buildpack at API version `0.7` should be considered invalid if the API version for that buildpack has been updated to `0.8`. Similarly, if a layer is changed from being a cache-only layer to being a cache and launch layer, then the cache should be considered invalid.
31-
32-
In addition to general cache invalidation conditions a buildpack should invalidate a previous layer if an important property changes, such as:
19+
buildpacks ensure byte-for-byte reproducibility of layers. File creation time is [normalized to January 1, 1980](https://medium.com/buildpacks/time-travel-with-pack-e0efd8bf05db) to ensure reproducibility. Byte-for-byte reproducibility means previous layers can be reused. However, we may want to invalidate previously cached layers if an important property changes, such as:
3320

3421
* the major version of the runtime changes eg: NodeJS changes from 16 to 18
3522
* requested application dependencies have changed eg: a Python application adds a dependency on the `requests` module
3623

37-
Launch layers are exported to an OCI registry and layer metadata is stored with the launch layer. The layer metadata is commonly used when deciding if a launch layer should be re-used from cache.
24+
Launch layers are exported to an OCI registry. The layer metadata is commonly used when deciding if a launch layer should be re-used. A launch layer may be re-used on an OCI registry without downloading the layer to the machine running a build.
3825

39-
## Strategies
26+
## Caching Strategies
4027

4128
Caching during the production of an application image is necessarily very flexible. Most buildpacks that wish to contribute a layer to the application image need only to
4229

@@ -48,7 +35,7 @@ This will guarantee that the previously published application image layer in the
4835

4936
Setting `build = true` makes a layer available to subsequent buildpacks. Therefore binaries installed to the `bin` directory on a `build = true` layer are available to subsequent buildpacks during the build phase. It is also the case that `lib` directories on a `build = true` later are added to the `LD_LIBRARY_PATH` during the build phase of subsequent buildpacks. Environment variables defined in a `build = true` layer are similarly available. For any layer where `launch = true` and `build = true`, a launch layer from the OCI registry can no longer be reused. Instead, the layer must be made available locally so that subsequent buildpacks can use it.
5037

51-
Setting `cache = true` allows additional fine-grained control over caching. The `cache = true` flag caches a layer and allows a buildpack to make _content_ level decisions about the validity of the cache (as opposed to using the less granular metadata). As an example, suppose a layer where `launch = true` installs a `jq` binary with version `1.5` and sets `version=1.5` in the layer metadata. By default, this layer will not be re-used from the registry when a buildpack requests `jq` with `version=1.6` to be installed. However, setting `cache = true` makes a previously built layer available during the build. A buildpack could then prefer to implement logic to restore `jq` with `version=1.5` instead of performing a potentially expensive download of `jq` with `version=1.6`. The `cache = true` setting allows for cache validation decisions to be made at a level of granularity that is much finer grained than layer metadata.
38+
Setting `cache = true` ensures that the layer is restored locally before the buildpacks build phase.
5239

5340
Setting `cache = false`, `build = false`, and `launch = true` is the most common configuration. If `cache = false`, `build = false`, and `launch = true` is not appropriate for your layer, then `cache = true`, `build = true`, and `launch = true` should be the next combination to evaluate:
5441

@@ -63,3 +50,5 @@ Other common configurations include
6350

6451
There are other boolean combinations of cache, build and launch. These provide significant flexibility in the caching system. Users of less common caching strategies need a good understanding of the [buildpacks specification on Layer Types](https://github.com/buildpacks/spec/blob/main/buildpack.md#layer-types
6552
).
53+
54+
The flexibility of buildpacks layer options allow fine-grained control over caching. A buildpack may make _content_ level decisions about the validity of a previous layer (as opposed to using the less granular metadata). A buildpack may contribute a launch layer that includes a built application and its dependencies. The same buildpack can also contribute a cache-only layer containing the source dependencies. In subsequent builds the buildpack can detect whether application dependencies have changed. The subset of dependencies that have changed may be updated on the cache layer. Then all dependencies may be restored from the cache layer and the built application is contributed as a new launch layer. In this way we make content-level decisions about the validity of dependencies. In addition, content-level caching strategies can save time and bandwidth by choosing to update only a subset of the cached content.

0 commit comments

Comments
 (0)