Skip to content

Commit 5da829f

Browse files
committed
Add draft RFC for adding CNBs
1 parent 4d33a26 commit 5da829f

File tree

1 file changed

+194
-0
lines changed

1 file changed

+194
-0
lines changed

toc/rfc/rfc-draft-add-cnbs.md

Lines changed: 194 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,194 @@
1+
# Meta
2+
[meta]: #meta
3+
- Name: Add Cloud Native Buildpacks
4+
- Start Date: 2023-05-01
5+
- Author: @gerg
6+
- Contributing Authors: @ryanmoran, @robdimsdale, @dsabeti, @aramprice, @dsboulder
7+
- Status: Draft
8+
- RFC Pull Request: #591
9+
10+
## Summary
11+
12+
[Cloud Native Buildpacks (CNBs)](https://buildpacks.io/), also known as v3
13+
buildpacks, are the current generation of buildpacks, and offer some
14+
improvements over the v2 buildpacks that CF Deployment currently uses. The
15+
Cloud Foundry Foundation already has an implementation of Cloud Native
16+
Buildpacks via the [Korifi](https://www.cloudfoundry.org/technology/korifi/)
17+
project, however these CNBs are not currently integrated with core CF
18+
Deployment.
19+
20+
This RFC proposes adding CNB support to Cloud Foundry and including the Korifi
21+
buildpacks in CF Deployment.
22+
23+
## Problem
24+
25+
The v2 buildpacks are effectively in maintenance mode and do not receive
26+
substantial new features. By not integrating with v3 buildpacks, Cloud Foundry
27+
is missing out on new capabilities like application
28+
[SBOM](https://en.wikipedia.org/wiki/Software_supply_chain) generation, new
29+
buildpacks like the Java Native and Web Servers CNBs, as well as any new
30+
features that are added to the still-actively-developed v3 buildpacks.
31+
32+
## Proposal
33+
34+
### Overview
35+
36+
Cloud Foundry integration with CNBs will be executed over at least two phases.
37+
38+
In the first phase, the Paketo buildpacks will be "shimmed" to work with Cloud
39+
Foundry's existing v2 buildpack interface. These shimmed Paketo buildpacks will
40+
be bundled with select [CNB lifecycle](https://github.com/buildpacks/lifecycle)
41+
binaries and lightweight orchestration binaries. At build time, the Cloud
42+
Foundry [buildpack lifecycle](https://github.com/cloudfoundry/buildpackapplifecycle)
43+
will invoke the orchestration binaries, which will map the v2 build process to
44+
the v3 build process.
45+
46+
In the second phase, unmodified Paketo buildpacks will be natively supported by
47+
Cloud Foundry. This will be implemented by integrating the buildpack shim logic
48+
into a Cloud Foundry lifecycle.
49+
50+
Depending on learnings from the first two phases of CNB integration or other
51+
factors, there may be additional phases of work, however these additional
52+
phases are not covered by this RFC.
53+
54+
Notably, this proposal does NOT include building or running OCI images on Cloud
55+
Foundry. This proposal is to use Paketo buildpacks to build and run Cloud
56+
Foundry droplets. This proposal does not preclude future efforts to integrate
57+
Cloud Foundry with OCI images, container registries, etc.
58+
59+
### Goals
60+
61+
1. Align CF Deployment with the main focus of the Buildpacks team
62+
1. Bring new build capabilities to CF Deployment users, including:
63+
1. [Application SBOMs](https://paketo.io/docs/howto/sbom/)
64+
1. [Web Servers Buildpack](https://github.com/paketo-buildpacks/web-servers)
65+
1. [Java Native Buildpack](https://github.com/paketo-buildpacks/java-native-image)
66+
1. Easier buildpack customization/composition
67+
1. Increase cohesion and application portability between CF Deployment and
68+
Korifi, via mutual Paketo integration
69+
1. Increase user base for the CNB lifecycle and Paketo buildpacks
70+
1. Open the door for eventual deprecation of the v2 buildpacks, reducing
71+
maintenance costs (v2 buildpack deprecation is NOT included in this RFC)
72+
73+
### High Level Implementation
74+
75+
In phase one, Cloud Foundry will produce a series of experimental shimmed
76+
buildpacks that include:
77+
1. A Paketo CNB
78+
1. Select executables from the CNB lifecycle
79+
1. Orchestrator executables that conform to the v2 buildpack interface
80+
81+
Cloud Foundry users will interact with the shimmed buildpacks the same as they
82+
do for any other buildpack. For example, the following commands will work as
83+
expected:
84+
85+
```
86+
$ cf create-buildpack ruby_cnb ./ruby_cnb_shim.zip
87+
...
88+
$ cf push my-app --buildpack=ruby_cnb
89+
```
90+
91+
At build time, the procedure will be as follows:
92+
1. The Cloud Foundry Buildpack App Lifecycle's Builder will invoke shim
93+
orchestrator executables, consistent with the v2 buildpack interface
94+
1. The shim orchestrator will invoke bundled CNB lifecycle executables
95+
1. The CNB lifecycle executables will invoke Paketo buildpack executables,
96+
consistent with the v3 buildpack interface
97+
1. The Paketo buildpack will build a set of "layer" directories
98+
1. The shim orchestrator will package the layer directories into a Cloud
99+
Foundry droplet and return it back to the Cloud Foundry Lifecycle
100+
101+
For a proof of concept of shimmed buildpacks, see @ryanmoran's
102+
[cfnb](https://github.com/ryanmoran/cfnb) project. Developing the shim
103+
externally at first will allow for more rapid iteration, without disrupting the
104+
core Cloud Foundry runtime.
105+
106+
During phase two, the shim logic will be integrated into the Cloud Foundry
107+
lifecycle itself, allowing for "native" Paketo buildpack support. The details
108+
of this integration will be decided once phase one is complete.
109+
110+
### Shim at Build Time
111+
112+
At build time, the shim orchestrator will expose the following v2 buildpack
113+
executables:
114+
115+
| Executable | Function |
116+
| --- | --- |
117+
| `detect` | Invoke the CNB Lifecycle `detector` executable |
118+
| `supply` | Invoke the CNB Lifecycle `builder` executable |
119+
| `finalize` | Add CNB `launcher` executable and runtime configuration files to build layers |
120+
| `release` | Package the generated layer directories into a Cloud Foundry droplet |
121+
122+
These executables will be invoked by the Cloud Foundry Buildpack App Lifecycle's
123+
builder.
124+
125+
### Shim at Run Time
126+
127+
At run time, the Cloud Foundry Buildpack App Lifecycle launcher will invoke the
128+
CNB Lifecycle `launcher` executable that was included in the droplet at build
129+
time. In order to support custom start commands for apps using the CNB
130+
buildpack shims, the CNB launcher invocation will be prepended to the app start
131+
command using the [`entrypoint_prefix` hook](https://github.com/cloudfoundry/buildpackapplifecycle/commit/29feb13caeff646f35585eee865c376c818fc2ea)
132+
in the CF Lifecycle launcher.
133+
134+
### Service Bindings
135+
136+
The largest anticipated breaking change from v2 to v3 buildpacks is the change
137+
in how service bindings are consumed by buildpacks. The v3 buildpacks expect
138+
service bindings to follow the [Kubernetes Service Binding specification](https://github.com/servicebinding/spec),
139+
which places service binding credentials on the filesystem. Cloud Foundry and
140+
v2 buildpacks instead use environment variables for service binding
141+
credentials.
142+
143+
In order to securely place credentials on the filesystem, Diego will need the
144+
ability to mount in-memory filesystems (e.g. `tmpfs`) in application
145+
containers. Once an in-memory filesystem is available, select `VCAP_SERVICES`
146+
environment variables can be translated into their Kubernetes Service Binding
147+
equivalents by the shim orchestrator, for consumption by the v3 buildpacks.
148+
149+
### Governance and Release
150+
151+
A `cnb-shim-builder` repository will be added to the Application Runtime
152+
Interfaces working group. This repository will contain the tooling necessary
153+
for converting existing Paketo buildpacks into shimmed CNBs.
154+
155+
The shimmed buildpacks will be experimental bosh releases, belonging to the
156+
Application Runtime Interfaces working group. The proposed list of buildpacks
157+
is as follows (in rough priority order):
158+
1. `java_native_cnb_release`
159+
1. `java_cnb_release`
160+
1. `web_servers_cnb_release`
161+
1. `dotnet_core_cnb_release`
162+
1. `nodejs_cnb_release`
163+
1. `python_cnb_release`
164+
1. `ruby_cnb_release`
165+
1. `procfile_cnb_release`
166+
1. `go_cnb_release`
167+
1. `nginx_cnb_release`
168+
1. `php_cnb_release`
169+
1. `rust_cnb_release`
170+
171+
It is not required to shim all buildpacks during phase 1 of this project.
172+
Depending on the effort to shim buildpacks and other factors, it may make sense
173+
to move to native CNB support before all Paketo buildpacks are shimmed. During
174+
phase 2, the above bosh release repositories will be converted to unshimmed
175+
bosh releases of the corresponding Paketo buildpacks.
176+
177+
The CNBs will initially be opt-in for CF Deployment operators via an
178+
experimental ops file. Once the buildpacks have reached a suitable level of
179+
maturity and stability, likely after phase 2, they will be added to the default
180+
set of buildpacks installed as part of CF Deployment.
181+
182+
### Stacks
183+
184+
Initially, the shimmed Paketo buildpacks will be compatible with `cflinuxfs4` to
185+
ease development and adoption. Paketo provides [multiple stacks](https://paketo.io/docs/concepts/stacks/#what-paketo-stacks-are-available)
186+
that are compatible with the Paketo buildpacks. There is an opportunity to
187+
adopt some or all of the Paketo stacks into CF Deployment, in addition to the
188+
buildpacks. The Paketo buildpacks have greater cohesion with the Paketo
189+
buildpacks than with `cflinuxfs4`, and the Paketo "base" and "tiny" stacks could
190+
offer security-conscious CF Deployment users stacks with fewer native
191+
dependencies than are currently included with `cflinuxfs4`. This RFC does not
192+
cover the adoption of additional stacks into CF Deployment, but it does open
193+
the door to add these stacks in the future.
194+

0 commit comments

Comments
 (0)