Separate task(s) for build-deps pre-fetching

[praiskup]

This logically separates the part that should be implemented in cachi2 in the future, per:
https://rpm-software-management.github.io/mock/feature-hermetic-builds

Fixe: #48

[praiskup]

Hmm, we can't download podman image in Pod:

subprocess.CalledProcessError: Command '['podman', 'pull', '--arch', 'amd64', 'registry.fedoraproject.org/fedora:rawhide@sha256:899da242bd0ae24e94b190de9a19b970af572136801c5a64010e6242cf27933e']' returned non-zero exit status 125.

[eskultety]

Yes, pretty much no podman commands will work inside a pod, that's the current nature of unprivileged nested containerization, you might have a better luck if you select buildah for the base image of the task, don't know exactly why but the implementation is different and so is more isolated, I'm definitely not an expert on what happens underneath, I'm just providing my recent experience.

[praiskup]

@eskultety we have a lockfile that refers to a container image that needs to be prefetched (as tarball so we can upload it to an MPC worker). What can we do to let cachi2 fetch it?

[eskultety]

@eskultety we have a lockfile that refers to a container image that needs to be prefetched (as tarball so we can upload it to an MPC worker). What can we do to let cachi2 fetch it?

@praiskup oh, tarballs are easy, you use our generic backend (aka glorified curl): https://hermetoproject.github.io/hermeto/generic/#specifying-artifacts-to-fetch, TL;DR you adopt a new artifacts.lock.yaml file to your repository and then it depends how you plan on interacting with hermeto (formerly cachi2):

1. if you invoke it directly in your task
   then you follow the doc I linked verbatim
2. if you depend on the default konflux prefetch task that's already in the catalog
   then you'd add something like {"type": "generic"} to your hermeto command line in your YAML pipeline definition

[eskultety]

@eskultety we have a lockfile that refers to a container image that needs to be prefetched (as tarball so we can upload it to an MPC worker). What can we do to let cachi2 fetch it?

@praiskup oh, tarballs are easy

@praiskup I take it back, sorry! I misread your message - there currently isn't a way how you could point hermeto to a container registry and expect a fetched tarball on the output.

[taylormadore]

@praiskup Can you use something like oras instead of podman to fetch the artifacts from the registry?

[eskultety]

@praiskup Can you use something like oras instead of podman to fetch the artifacts from the registry?

There's also a konflux-ci build of it: https://github.com/konflux-ci/oras-container

[praiskup]

I think I can fetch images with oras. But can I then extract the directory tree out of it? Like here?
https://github.com/rpm-software-management/mock/blob/840c438abba6c3a5adb2f082261561a36619b5eb/mock/py/mockbuild/podman.py#L216-L225

[eskultety]

I think I can fetch images with oras. But can I then extract the directory tree out of it? Like here? https://github.com/rpm-software-management/mock/blob/840c438abba6c3a5adb2f082261561a36619b5eb/mock/py/mockbuild/podman.py#L216-L225

Good question. Looks like that's not possible, I just tried to copy a random image from dockerhub, but I only got a bunch of layers, so yeah, not trivially mounted.

[praiskup]

Somewhat related:
rpm-software-management/mock#1642
containers/podman#27323