proposal: Load image from file

[wtraylor]

The Problem

A reproducible project should be self-contained, that is contain all files and dependencies necessary to re-execute it.

For that reason I would like to include my Docker image file in the repository. I don’t want to rely on Docker Hub as an external dependency.

Currently, Popper allows me to specify a custom Dockerfile, but that in turn inevitably relies on a base layer from a Docker registry like Docker Hub.

My Current Workaround

I save my base image to a file and add it with Git-LFS to the repo:

docker image save debian:stable-20201012-slim --output image.tar
gzip image.tar
git lfs track '*.tar.gz'
git add image.tar.gz

Then I load the image in the first step of the Popper workflow:

steps:
- uses: "sh"
  runs: ["docker"]
  args: ["image", "load",  "--input", "image.tar.gz"]

That fails if Popper runs itself in a container (see #956).

Suggestion

Allow a file in the uses: field like this:

steps:
- uses: "image.tar.gz"
  runs: ["echo"]
  args: ["Hello World!"]

For Singularity it is natural to load an image from file, and Docker provides docker image load. However, I don’t know how easy it would be to run Singularity from a Docker image file or vice versa.

[ivotron]

thanks a lot for bringing this up @wtraylor. Similarly to what was mentioned in #956 (comment), this could be solved by doing:

steps:
- uses: docker://docker:20.10.1
  args: ["image", "load",  "--input", "image.tar.gz"]

In general, there is a portability problem with this approach though. The above is specific to docker, and as you mentioned it wouldn't work on other container runtimes.

So one alternative to address this would be to allow file-based images to be referenced in the new syntax that will be introduced by #907. This new feature that will be implemented as part of #907 has the goal of abstracting the underlying engine with regards to the manipulation of images. As long as the tarball is an OCI-compliant image, and as long as the underlying engine supports OCI images, this would work in an engine-agnostic way.