Support pulling container images with excessive layers

[arewm]

Feature request description

I would like to build images using source containers in order to improve usability. In this use case, I would clone the git repo and prefetch all dependencies, storing each item as its own layer in an OCI artifact (to improve data storage in the image registry). I would then like to reference this container in a Tekton task, mounting the image as a file system for a future build.

Suggest potential solution

If the remote image exceeds the supported layers, podman can intuitively pull the image down to the filesytem

Have you considered any alternatives?

An alternative would be to skopeo copy the image to a specific directory.

Additional context

No response

[rhatdan]

This is not an issue, but a discussion.

Don't really understand what you want to do here. But it sounds like you want to do some kind of gentoo type of container image?

[arewm]

I am not familiar with the background of a gentoo type of container image. Is there more detail on that to determine how similar it might be?

[rhatdan]

I am asking you to more clearly define what you want?

Overlay file system has a kernel limit controlling how many layers may be used.

[arewm]

As I am aware of it, OCI artifacts do not have a limit to the number of layers that they can contain. The limit is from the file system as you mention. One of the use cases to leverage these images is just to scopeo copy the contents to the local file system and then inspect them after copied/extracted, i.e.

$ skopeo copy docker://registry.access.redhat.com/ubi9/ubi-minimal:9.2-750.1697625013-source dir:$HOME/foo
Getting image source signatures
Copying blob 0e8bb64f6235 done   |
Copying blob b8c85a26a7eb done   |
Copying blob 07672579645b done   |
Copying blob 14a66440c342 done   |
Copying blob c2cc285526e0 done   |
Copying blob 5932f335962d done   |
Copying blob c3db799dbda7 done   |
Copying blob 1b556f1ad153 done   |
[...]

While these source containers might just be created for compliance requirements where dedicated inspection tooling can be created, integrating these many-layered images becomes harder if you do not want to rely on multiple cli tools.

The specific use case that I am considering is having a single OCI artifact which contains all build sources for a container (git repo + resolved dependencies) which can then be used as a source to generate the container image binary in a reproducible manner. Since podman cp requires a local container, it is not possible to use this to extract the contents due to the fact that there are too many layers for the file system. While podman export can be potentially used, the archive then needs to be extracted.

I wanted to be able to leverage a single CLI for leveraging a build source container and also building the container image. It seems like this might already be possible using buildah though ...

$ buildah unshare
# cd $(buildah mount $(buildah from registry.access.redhat.com/ubi9/ubi-minimal:9.2-750.1697625013-source))
Trying to pull registry.access.redhat.com/ubi9/ubi-minimal:9.2-750.1697625013-source...
Getting image source signatures
Checking if image destination supports signatures
[...]

I can use buildah unshare, cd to the mounted directory, and then build the container image from there.

[rhatdan]

If skopeo can handle this, why do you want Podman to do it?

[arewm]

I wanted a single tool to be able to be used for building these containers. Now that I know that buildah can do it, I don't think Podman needs to be able to do it.

[arewm]

Based on my discovery of being able to achieve this with buildah instead, I'll close this discussion. Thanks.