Merge pull request #665 from buildpacks/breakup-buildpack-api

Break up Buildpack API section into separate "how to" pages

Author: AidanDelaney

content/docs/.common/concepts/extension.md

@@ -34,7 +34,7 @@ provide. By contrast, Dockerfiles are the most-used and best-understood mechanis
 installing OS-level dependencies for containers.
 
 The CNB Dockerfiles feature allows Dockerfiles to "provide" dependencies that buildpacks "require" through a
-shared [build plan](/docs/reference/spec/buildpack-api/#build-plan), by introducing the concept of image extensions.
+shared [build plan], by introducing the concept of image extensions.
 
 ## What do they look like?
 
@@ -67,7 +67,7 @@ The extension determines if it is needed or not.
 
 Like buildpacks, extensions participate in the `detect` phase - analyzing application source code to determine if they
 are needed. During `detect`, extensions can contribute to
-the [build plan](/docs/reference/spec/buildpack-api/#build-plan) - recording dependencies that they are able to "
+the [build plan] - recording dependencies that they are able to "
 provide" (though unlike buildpacks, they can't "require" anything).
 
 If the provided order contains extensions, the output of `detect` will be a group of image extensions and a group of
@@ -80,3 +80,5 @@ The extension outputs Dockerfiles that can be used to extend either or both of t
 
 For more information and to see a build in action,
 see [authoring an image extension](/docs/for-buildpack-authors/tutorials/basic-extension).
+
+[build plan]: /docs/for-buildpack-authors/how-to/write-buildpacks/use-build-plan

content/docs/for-buildpack-authors/concepts/caching-strategies.md

@@ -14,7 +14,7 @@ There are three types of layers that can be contributed to an image
 
 * `build` layers -- the directory will be accessible by subsequent buildpacks,
 * `cache` layers -- the directory will be included in the cache,
-* `launch` layers -- the directory will be included in the run image as a single layer,
+* `launch` layers -- the directory will be included in the final app image as a single layer,
 
 In this section we look at caching each layer type.
 

content/docs/for-buildpack-authors/how-to/distribute-buildpacks/craft-order.md

@@ -0,0 +1,10 @@
++++
+title="Craft a buildpack order"
+weight=99
++++
+
+<!--more-->
+
+This page is a stub! The CNB project is applying to [Google Season of Docs](https://developers.google.com/season-of-docs/docs/timeline) to receive support for improving our documentation. Please check back soon.
+
+If you are familiar with this content and would like to make a contribution, please feel free to open a PR :)

content/docs/for-buildpack-authors/how-to/distribute-buildpacks/package-buildpack.md

@@ -4,7 +4,7 @@ title="Package a buildpack or extension"
 aliases=[
   "/docs/buildpack-author-guide/package-a-buildpack"
 ]
-weight=5
+weight=4
 summary="Learn how to package your buildpack or extension for distribution."
 +++
 

content/docs/for-buildpack-authors/how-to/write-buildpacks/add-sbom.md

@@ -1,6 +1,6 @@
 +++
 title="Provide a Software Bill-of-Materials"
-weight=2
+weight=5
 +++
 
 Buildpacks can provide a [Software `Bill-of-Materials`](https://en.wikipedia.org/wiki/Software_bill_of_materials) (SBOM)

content/docs/for-buildpack-authors/how-to/write-buildpacks/create-layer.md

@@ -1,10 +1,50 @@
 +++
 title="Create dependency layers"
-weight=99
+weight=3
 +++
 
+Each directory created by the buildpack under the `CNB_LAYERS_DIR` can be used as a layer in the final app image or build cache.
+
 <!--more-->
 
-This page is a stub! The CNB project is applying to [Google Season of Docs](https://developers.google.com/season-of-docs/docs/timeline) to receive support for improving our documentation. Please check back soon.
+That is, each directory can be used for any of the following purposes:
+
+| Layer Type |                                                                                                             |
+|------------|-------------------------------------------------------------------------------------------------------------|
+| `Launch`   | the directory will be included in the **final app image** as a single layer                                 |
+| `Cache`    | the directory will be included in the **build cache** and restored to the `CNB_LAYERS_DIR` on future builds |
+| `Build`    | the directory will be accessible to **buildpacks that follow** in the build (via the environment)           |
+
+A buildpack can control how a layer will be used by creating a `<layer>.toml` with a name matching the directory it describes in the `CNB_LAYERS_DIR`.
+
+### Example
+
+A buildpack might create a `$CNB_LAYERS_DIR/python` directory and a `$CNB_LAYERS_DIR/python.toml` with the following contents:
+
+```
+launch = true
+cache = true
+build = true
+```
+
+In this example:
+* the final app image will contain a layer with `python`, as this is needed to run the app
+* the `$CNB_LAYERS_DIR/python` directory will be pre-created for future builds, avoiding the need to re-download this large dependency
+* buildpacks that follow in the build will be able to use `python`
+
+### Example
+
+This is a simple `./bin/build` script for a buildpack that runs Python's `pip` package manager to resolve dependencies:
+
+```
+#!/bin/sh
+
+PIP_LAYER="$CNB_LAYERS_DIR/pip"
+mkdir -p "$PIP_LAYER/modules" "$PIP_LAYER/env"
+
+pip install -r requirements.txt -t "$PIP_LAYER/modules" \
+  --install-option="--install-scripts=$PIP_LAYER/bin" \
+  --exists-action=w --disable-pip-version-check --no-cache-dir
 
-If you are familiar with this content and would like to make a contribution, please feel free to open a PR :)
+echo "launch = true" > "$PIP_LAYER.toml"
+```

content/docs/for-buildpack-authors/how-to/write-buildpacks/get-started.md

@@ -0,0 +1,111 @@
++++
+title="Get started"
+weight=1
++++
+
+To write a buildpack, we follow the [Buildpack Specification](https://github.com/buildpacks/spec/blob/main/buildpack.md),
+which defines the contract between buildpacks and the lifecycle.
+
+<!--more-->
+
+A buildpack must contain three files:
+
+* `buildpack.toml`
+* `bin/detect`
+* `bin/build`
+
+The two files in `bin/` must be executable.
+They can be shell scripts written in a language like Bash,
+or they can be executables compiled from a language like Go.
+
+## `buildpack.toml`
+
+A buildpack must contain a `buildpack.toml` file in its root directory.
+
+### Example
+
+```
+api = "0.10"
+
+[buildpack]
+id = "example.com/python"
+version = "1.0"
+
+# Targets the buildpack will work with
+[[targets]]
+os = "linux"
+
+# Stacks (deprecated) the buildpack will work with
+[[stacks]]
+id = "io.buildpacks.stacks.jammy"
+```
+
+For more information, see [buildpack config](/docs/reference/config/buildpack-config).
+
+## `bin/detect`
+
+### Usage
+
+```
+bin/detect
+```
+
+### Summary
+
+`bin/detect` is used to determine if a buildpack can work with a given codebase.
+It will often check for the existence of a particular file,
+or some configuration indicating what kind of application has been provided.
+
+Two environment variables identify important file system paths:
+
+* `CNB_PLATFORM_DIR` - a directory containing platform provided configuration, such as environment variables.
+* `CNB_BUILD_PLAN_PATH` - a path to a file containing the [build plan].
+
+In addition, the working directory for `bin/detect` is the application directory.
+
+`bin/detect` must return an exit code of `0` if the codebase can be serviced by this buildpack,
+and `100` if it cannot.
+Other exit codes indicate an error during detection.
+
+### Example
+
+This is a simple example of a buildpack that detects a Python application
+by checking for the presence of a `requirements.txt` file:
+
+```
+#!/bin/sh
+
+if [ -f requirements.txt ]; then
+  echo "Python Buildpack"
+  exit 0
+else
+  exit 100
+fi
+```
+
+## `bin/build`
+
+### Usage
+
+```
+bin/build
+```
+
+`bin/build` does (all or part of) the work of transforming application source code into a runnable artifact.
+It will often resolve dependencies, install binary packages, and compile code.
+Three environment variables identify important file system paths:
+
+* `CNB_LAYERS_DIR` - a directory that may contain subdirectories representing each layer created by the buildpack in the final image or build cache.
+* `CNB_PLATFORM_DIR` - a directory containing platform provided configuration, such as environment variables.
+* `CNB_BP_PLAN_PATH` - a path to a file containing the [build plan].
+
+In addition, the working directory for `bin/build` is the application directory.
+
+All changes to the codebase in the working directory will be persisted in the final image,
+along with any launch layers created in the `CNB_LAYERS_DIR`.
+
+It is important to note that multiple buildpacks may work together to create the final image,
+each contributing a subset of the dependencies or configuration needed to run the application.
+In this way, buildpacks are modular and composable.
+
+[build plan]: /docs/for-buildpack-authors/how-to/write-buildpacks/use-build-plan

content/docs/for-buildpack-authors/how-to/write-buildpacks/specify-launch-processes.md

@@ -1,6 +1,6 @@
 +++
 title="Specify process types"
-weight=1
+weight=4
 +++
 
 One of the benefits of buildpacks is that they are multi-process - an image can have multiple entrypoints for each operational mode.

content/docs/for-buildpack-authors/how-to/write-buildpacks/use-build-plan.md

@@ -1,10 +1,121 @@
 +++
 title="Use the build plan"
-weight=99
+weight=2
 +++
 
+The [Build Plan](https://github.com/buildpacks/spec/blob/main/buildpack.md#build-plan-toml) is a document that buildpacks can use to pass information between the `detect` and `build` phases, and between each other.
+The build plan is passed (by the lifecycle) as a parameter to the `detect` and `build` binaries of each buildpack.
+
 <!--more-->
 
-This page is a stub! The CNB project is applying to [Google Season of Docs](https://developers.google.com/season-of-docs/docs/timeline) to receive support for improving our documentation. Please check back soon.
+During the `detect` phase, each buildpack may write something it `requires` or `provides` (or both) into the Build Plan.
+A buildpack can `require` or `provide` multiple dependencies, and even multiple groupings of dependencies (using `or` lists).
+Additionally, multiple buildpacks may `require` or `provide` the same dependency.
+For detailed information, consult the [spec](https://github.com/buildpacks/spec/blob/main/buildpack.md#buildpack-plan-toml).
+
+The lifecycle uses the Build Plan to determine whether a particular list of buildpacks can work together,
+by seeing whether all dependencies required can be provided by that list.
+
+Later, during the `build` phase, each buildpack may read the Buildpack Plan (a condensed version of the Build Plan, composed by the lifecycle) to determine what it should do.
+
+Let's see how this works with an example.
+
+### Example: `node-engine` buildpack
+
+Let's walk through some possible cases a `node-engine` buildpack may consider:
+
+1.  Nothing in the app explicitly calls out that it is needed
+2.  It is explicitly referred to in some configuration file
+
+We will also consider what an `NPM` and a `JVM` buildpack may do.
+
+#### Scenario 1: No Explicit Request
+
+A `node-engine` buildpack is always happy to `provide` the `node` dependency. The build plan it will write may look something like:
+```
+[[provides]]
+name = "node"
+```
+> **NOTE:** If this was the only buildpack running, this would fail the `detect` phase. In order to pass, every `provides` must be matched up with a `requires`, whether in the same buildpack or in another buildpack.
+> See the [spec](https://github.com/buildpacks/spec/blob/main/buildpack.md#phase-1-detection) for particulars on how ordering buildpacks can adjust detection results.
+
+#### Scenario 2: One Version Requested
+
+During the `detect` phase, the `node-engine` buildpack sees in one configuration file (e.g. a `.nvmrc` file in the app directory) that `node v10.x` is explicitly requested by the application. Seeing that, it may write the below text to the build plan:
+```
+[[provides]]
+name = "node"
+
+[[requires]]
+name = "node"
+version = "10.x"
+
+[requires.metadata]
+version-source = ".nvmrc"
+```
+
+As always, the buildpack `provides` `node`. In this particular case, a version of `node` (`10.x`) is being requested in a configuration file (`.nvmrc`). The buildpack chooses to add an additional piece of metadata (`version-source`), so that it can understand where that request came from.
+
+#### NPM Buildpack
+
+`NPM` is the default package manager for `node`. A NPM Buildpack may ensure that all the packages for the application are present (by running `npm install`), and perhaps cache those packages as well, to optimize future builds.
+
+NPM is typically distributed together with node. As a result, a NPM buildpack may require `node`, but not want to `provide` it, trusting that the `node-engine` buildpack will be in charge of `providing` `node`.
+
+The NPM buildpack could write the following to the build plan, if the buildpack sees that `npm` is necessary (e.g., it sees a `package.json` file in the app directory):
+```
+[[requires]]
+name = "node"
+```
+
+If, looking in the `package.json` file, the NPM buildpack sees a specific version of `node` requested in the [engines](https://docs.npmjs.com/files/package.json#engines) field (e.g. `14.1`), it may write the following to the build plan:
+```
+[[requires]]
+name = "node"
+version = "14.1"
+
+[requires.metadata]
+version-source = "package.json"
+```
+
+> **NOTE:** As above, if this was the only buildpack running, this would fail the `detect` phase. In order to pass, every `provides` must be matched up with a `requires`, whether in the same buildpack or in another buildpack.
+> See the [spec](https://github.com/buildpacks/spec/blob/main/buildpack.md#phase-1-detection) for particulars on how ordering buildpacks can adjust detection results.
+
+However, if the NPM Buildpack was run together with the Node Engine buildpack (which `provides` `node`), the lifecycle will see that all requirements are fulfilled, and select that group as the correct set of buildpacks.
+
+### Example: JVM buildpack
+
+Java is distributed in two formats - the `jdk` (Java Development Kit), which allows for compilation and running of Java programs, and the `jre` (Java Runtime Environment,  which allows for running compiled Java programs).
+A very naive implementation of the buildpack may have it write several `provides` options to the build plan, detailing everything that it can provide,
+while later buildpacks would figure out based on the application which options it requires, and would `require` those.
+In this particular case, we can use the `or` operator to present different possible build plans the buildpack can follow:
+
+```
+# option 1 (`jre` and `jdk`)
+[[provides]]
+name = "jre"
+
+[[provides]]
+name = "jdk"
+
+# option 2 (or just `jdk`)
+[[or]]
+[[or.provides]]
+name = "jdk"
+
+# option 3 (or just `jre`)
+[[or]]
+[[or.provides]]
+name = "jre"
+```
+
+The buildpack gives three options to the lifecycle:
+* It can provide a standalone `jre`
+* It can provide a standalone `jdk`
+* It can provide both the `jdk` and `jre`
+
+As with the other buildpacks, this alone will not be sufficient for the lifecycle. However, other buildpacks that follow may `require` certain things.
 
-If you are familiar with this content and would like to make a contribution, please feel free to open a PR :)
+For example, another buildpack may look into the application and, seeing that it is a Java executable, `require` the `jre` in order to run it.
+When the lifecycle analyzes the results of the `detect` phase, it will see that there is a buildpack which provides `jre`, and a buildpack that requires `jre`,
+and will therefore conclude that those options represent a valid set of buildpacks.

content/docs/for-buildpack-authors/tutorials/basic-buildpack/04_build-app.md

@@ -110,4 +110,4 @@ A new image named `test-node-js-app` was created in your Docker daemon with a la
 <a href="/docs/for-buildpack-authors/tutorials/basic-buildpack/05_make-app-runnable" class="button bg-pink">Next Step</a>
 <!--+end+-->
 
-[layers-dir]: /docs/reference/spec/buildpack-api/#layers
+[layers-dir]: /docs/for-buildpack-authors/how-to/write-buildpacks/create-layer