Skip to content

Implement RFC #0131 - Build Observability telemetry specification (Platform spec) #421

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Copilot wants to merge 7 commits into
base: platform/0.16
Choose a base branch
from
Open

Implement RFC #0131 - Build Observability telemetry specification (Platform spec) #421

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
9a7d334
Initial plan
Copilot Nov 20, 2025
e0cd8ee
Add telemetry specification for RFC #0131 - Build Observability
Copilot Nov 20, 2025
7dfb4c8
Add telemetry outputs and inputs to buildpack specification
Copilot Nov 20, 2025
cf88fca
Add telemetry output to image extension specification
Copilot Nov 20, 2025
a75f3f3
Use CNB_OTEL_LOG_PATH instead of hardcoded telemetry paths
Copilot Nov 24, 2025
5ac3690
Add MAY write telemetry clause to detect and build phases
Copilot Nov 24, 2025
17e6edc
Revert buildpack.md and image_extension.md changes
Copilot Dec 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions buildpack.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,7 @@ Executable: `/bin/detect`, Working Dir: `<app[AR]>`
| `$CNB_BUILD_PLAN_PATH` | E | Absolute path of the build plan |
| `$CNB_BUILDPACK_DIR` | ER | Absolute path of the buildpack root directory |
| `$CNB_EXEC_ENV` | AR | Target execution environment ("production", "test", "development") |
| `$CNB_OTEL_TRACEPARENT` | AR | W3C Trace Context traceparent for trace correlation (optional) |
| `$CNB_PLATFORM_DIR` | AR | Absolute path of the platform directory |
| `$CNB_PLATFORM_DIR/env/` | AR | User-provided environment variables for build |
| `$CNB_PLATFORM_DIR/#` | AR | Platform-specific extensions |
Expand All @@ -201,6 +202,7 @@ Executable: `/bin/detect`, Working Dir: `<app[AR]>`
| Standard output | Logs (info) |
| Standard error | Logs (warnings, errors) |
| `$CNB_BUILD_PLAN_PATH` | Contributions to the the Build Plan (TOML) |
| `$CNB_LAYERS_DIR/tracing/buildpacks/<id>@<version>-detect.jsonl` | Buildpack telemetry in [OpenTelemetry File Exporter format](https://opentelemetry.io/docs/specs/otel/protocol/file-exporter/) (optional) |

### Build

Expand All @@ -213,6 +215,7 @@ Executable: `/bin/build`, Working Dir: `<app[AI]>`
| `$CNB_BP_PLAN_PATH` | ER | Relevant [Buildpack Plan entries](#buildpack-plan-toml) from detection (TOML) |
| `$CNB_BUILDPACK_DIR` | ER | Absolute path of the buildpack root directory |
| `$CNB_EXEC_ENV` | AR | Target execution environment ("production", "test", "development") |
| `$CNB_OTEL_TRACEPARENT` | AR | W3C Trace Context traceparent for trace correlation (optional) |
| `$CNB_PLATFORM_DIR` | AR | Absolute path of the platform directory |
| `$CNB_PLATFORM_DIR/env/` | AR | User-provided environment variables for build |
| `$CNB_PLATFORM_DIR/#` | AR | Platform-specific extensions |
Expand Down Expand Up @@ -240,6 +243,7 @@ Executable: `/bin/build`, Working Dir: `<app[AI]>`
| `$CNB_LAYERS_DIR/<layer>/env.launch/<process>/` | Env vars for launch (after `env`, before `exec.d`) for the launched process |
| `$CNB_LAYERS_DIR/<layer>/env.build/` | Env vars for subsequent buildpacks (after `env`) |
| `$CNB_LAYERS_DIR/<layer>/*` | Other content for launch and/or subsequent buildpacks |
| `$CNB_LAYERS_DIR/tracing/buildpacks/<id>@<version>-build.jsonl` | Buildpack telemetry in [OpenTelemetry File Exporter format](https://opentelemetry.io/docs/specs/otel/protocol/file-exporter/) (optional) |
Copy link
Copy Markdown
Contributor

@edmorley edmorley Nov 20, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused by this.

From reading the RFC, I had thought the new tracing/ subdirectory was going at the very root of the layers dir hierarchy (ie /layers/tracing/...), hence the buildpack name and ID being included the the filenames of the .jsonl files.

However, from the line above the one added here there is:

| $CNB_LAYERS_DIR/<layer>/* | Other content for launch and/or subsequent buildpacks

...which would suggest that $CNB_LAYERS_DIR is instead not /layers but /layers/<buildpack_id>, given that buildpack layers gets nested under a new directory for each buildpack? And if that were the case, then the resultant new tracing files location would resolve to /layers/<buildpack_id>/tracing/... which doesn't seem to match the spec?

Or does it just mean there is a typo in the original content of the spec here?

ie Should the existing line above instead say:

| $CNB_LAYERS_DIR/<buildpack_id>/<layer>/* | Other content for launch and/or subsequent buildpacks

If there is an existing typo, then it's unrelated to this PR (but we should fix it at some point), but it would at least explain the difference here. However, if there isn't an existing typo, then I'm wondering if the new addition for this PR is correct?

Copy link
Copy Markdown
Contributor

@jabrown85 jabrown85 Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good spot.

I looked earlier and thought CNB_LAYERS_DIR was the /layers in the context of both platform and buildpack - because it is /layers on the produced image. CNB_LAYERS_DIR=/layers is what is exported on the image and what is used for lifecycle itself during execution.

It makes sense though, looking at buildpack spec, that CNB_LAYERS_DIR is actually /layers/<buildpack-id> in the context of a buildpack execution. lifecycle mutates this in the execution context of each buildpack.

The spec isn't released yet so we have time to adjust this. I don't think buildpacks have access to anything that represents /layers though.

WDYT @hone @edmorley @joshwlewis?

Copy link
Copy Markdown
Member

@jkutner jkutner Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea, i think these are both true:

  • We want the tracing dir in /layers/<buildpack-id>
  • The CNB_LAYERS_DIR is /layers/<buildpack-id>

If those are true, then I think we have it right?

Copy link
Copy Markdown
Contributor

@edmorley edmorley Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want the tracing dir in /layers/<buildpack-id>

The RFC says says this isn't the case though?

The structure has generic things at the top (eg lifecycle otel files), and then the buildpack files are named with the ID etc.

Copy link
Copy Markdown
Member

@jkutner jkutner Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, i see now. Here's one proposal to resolve this (but I also want to see an alternative from Jesse):

Location

Tracing files will be written to different locations depending on their source.

All lifecycle execution traces SHOULD be written to /layers/tracing/lifecycle/{phase}.jsonl.
Buildpack traces MAY be written to${CNB_LAYERS_DIR}/tracing-{phase}.jsonl.
Extension traces MAY be written to/layers/tracing/extensions/{id}@{version}-{phase}.jsonl.

A completed build with tracing might have a tracing file hierarchy like this:

<layers>
├── tracing
    ├── extensions
    │   ├── some-id@some-version-detect.jsonl
    │   └── some-id@some-version-generate.jsonl
    └── lifecycle
        ├── analyze.jsonl
        ├── build.jsonl
        ├── detect.jsonl
        ├── export.jsonl
        ├── extend.jsonl
        └── restore.jsonl
└── <buildpack-id>
    ├── tracing-build.jsonl
    └── tracing-detect.jsonl

The drawback of this proposal is that we are polluting the CNB_LAYERS_DIR with non-layer information.

Copy link
Copy Markdown
Member

@jkutner jkutner Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should go with Jesse's proposal as it seems to be cleaner within lifecycle

Copy link
Copy Markdown
Contributor

@jabrown85 jabrown85 Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYT @edmorley @joshwlewis?

Copy link
Copy Markdown
Contributor

@jabrown85 jabrown85 Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot update the spec with the following changes

  • In the buildpack spec, CNB_OTEL_LOG_PATH is passed to buildpacks in each phase for them to write to when present. Buildpacks will no longer use CNB_LAYERS_DIR to build their own path to write jsonl
  • The extension spec will also change to allow them to write to CNB_OTEL_LOG_PATH and not rely on a static path in the spec.
  • In the platform spec, CNB_OTEL_LOG_PATH should be passed to buildpacks and point to a $CNB_LAYERS_DIR/tracing/buildpacks/<buildpack>@<version>-<phase>.jsonl path created by the platform. Similar changes to extensions.

Copy link
Copy Markdown
Contributor

@jabrown85 jabrown85 Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do the spec changes look @edmorley @jkutner ? I introduced the CNB_OTEL_LOG_PATH

Copy link
Copy Markdown
Member

@jkutner jkutner Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm


### Exec.d

Expand Down
1 change: 1 addition & 0 deletions image_extension.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@ Correspondingly, each `/bin/generate` executable:
- `context.run`
- `context.build`
- MAY write key-value pairs to `<output>/extend-config.toml` that are provided as build args to build.Dockerfile when extending the build image.
- MAY write telemetry data to `<layers>/tracing/extensions/<id>@<version>-detect.jsonl` or `<layers>/tracing/extensions/<id>@<version>-generate.jsonl` in [OpenTelemetry File Exporter format](https://opentelemetry.io/docs/specs/otel/protocol/file-exporter/).
- MUST NOT write SBOM (Software-Bill-of-Materials) files as described in the [Software-Bill-of-Materials](#software-bill-of-materials) section.

#### Context Folders
Expand Down
75 changes: 75 additions & 0 deletions platform.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,11 @@ Examples of a platform might include:
- [Run Image Resolution](#run-image-resolution)
- [Registry Authentication](#registry-authentication)
- [Experimental Features](#experimental-features)
- [Telemetry](#telemetry)
- [Telemetry Files](#telemetry-files)
- [Telemetry File Properties](#telemetry-file-properties)
- [Context Propagation](#context-propagation)
- [Telemetry Consumption](#telemetry-consumption)
- [Buildpacks](#buildpacks)
- [Buildpacks Directory Layout](#buildpacks-directory-layout)
- [Image Extensions](#image-extensions)
Expand All @@ -74,6 +79,7 @@ Examples of a platform might include:
- [POSIX Path Variables](#posix-path-variables)
- [User-Provided Variables](#user-provided-variables)
- [Operator-Defined Variables](#operator-defined-variables)
- [Platform-Provided Variables](#platform-provided-variables)
- [Launch Environment](#launch-environment)
- [Caching](#caching)
- [Build Reproducibility](#build-reproducibility)
Expand Down Expand Up @@ -1141,6 +1147,65 @@ When an experimental feature is invoked, the lifecycle:
- SHALL fail if `CNB_EXPERIMENTAL_MODE=error`
- SHALL continue without warning if `CNB_EXPERIMENTAL_MODE=silent`

### Telemetry

The lifecycle and buildpacks MAY emit telemetry data during build operations to enable observability of build performance and behavior.

#### Telemetry Files

Telemetry data MUST be written using the [OpenTelemetry File Exporter format](https://opentelemetry.io/docs/specs/otel/protocol/file-exporter/) in JSONL format.

Telemetry files MUST be written to the following locations under `<layers>/tracing/`:
- Lifecycle telemetry: `<layers>/tracing/lifecycle/<phase>.jsonl`
- Where `<phase>` is one of: `analyze`, `detect`, `restore`, `extend`, `build`, `export`
- Buildpack telemetry: `<layers>/tracing/buildpacks/<id>@<version>-<phase>.jsonl`
- Where `<id>` is the buildpack ID, `<version>` is the buildpack version, and `<phase>` is either `detect` or `build`
- Extension telemetry: `<layers>/tracing/extensions/<id>@<version>-<phase>.jsonl`
- Where `<id>` is the extension ID, `<version>` is the extension version, and `<phase>` is either `detect` or `generate`

Example telemetry file hierarchy:
```
<layers>
└── tracing
├── buildpacks
│ ├── example-buildpack@1.0.0-detect.jsonl
│ └── example-buildpack@1.0.0-build.jsonl
├── extensions
│ └── example-extension@1.0.0-generate.jsonl
└── lifecycle
├── analyze.jsonl
├── detect.jsonl
├── restore.jsonl
├── build.jsonl
└── export.jsonl
```

#### Telemetry File Properties

Telemetry files:
- MAY be written at any point during the build to ensure telemetry is persisted in case of failures, process terminations, or crashes
- MUST NOT be truncated or deleted during the build to allow telemetry processing by the platform during or after the build
- MUST NOT be included in the exported app image
- MUST be group-readable to allow analysis by the user and/or platform
- SHOULD only be read and written by the component that created them (lifecycle or individual buildpack/extension)

#### Context Propagation

The platform MAY set `CNB_OTEL_TRACEPARENT` in the lifecycle execution environment to enable correlation of lifecycle and buildpack traces with platform traces.

When `CNB_OTEL_TRACEPARENT` is provided:
- The value MUST conform to the [W3C Trace Context specification for traceparent field values](https://www.w3.org/TR/trace-context/#traceparent-header-field-values)
- The lifecycle and buildpacks SHOULD use the provided `trace-id` and `parent-id` when generating traces

#### Telemetry Consumption

The platform MAY read telemetry files during or after the build for observability purposes.

Telemetry files:
- MUST NOT be used as an API, contract, or communication mechanism between buildpacks or between the lifecycle and buildpacks
- MUST NOT contain personally identifiable information (e.g., usernames, email addresses, IP addresses)
- MUST NOT contain business-sensitive information (e.g., passwords, access keys, image names, URLs, source code repository names)

## Buildpacks

### Buildpacks Directory Layout
Expand Down Expand Up @@ -1220,6 +1285,16 @@ however the operator-defined value is always applied after the buildpack-provide

The platform SHOULD NOT set operator-provided environment variables directly in the lifecycle execution environment.

##### Platform-Provided Variables

The platform MAY set the following environment variables in the lifecycle execution environment:

| Env Variable | Description |
|-------------------------|-------------------------------------------------------------------------------------------------------|
| `CNB_OTEL_TRACEPARENT` | W3C Trace Context traceparent value for correlating lifecycle and buildpack traces with platform traces. See [Telemetry](#telemetry). |

These variables SHALL be directly inherited by buildpacks without modification.

#### Launch Environment

User-provided modifications to the process execution environment SHOULD be set directly in the lifecycle execution environment.
Expand Down