Documentation missing: How to run a local registry? (additionally I get an error)

[dirk68-fu]

The documentation and the conference videos don't show how to run a local registry which swift-container-plugin needs.

From googling and from issue #68 I was able to figure out that running

docker run -d -p 5555:5000 registry:2

should do the trick.

Unfortunately I get the following error when swift-container-plugin runs:

swift run containertool .build/x86_64-swift-linux-musl/debug/helloServer --repository localhost:5555/hello -v
Building for debugging...
[1/1] Write swift-version--58304C5D6DBC2206.txt
Build of product 'containertool' complete! (0.26s)
Base image architecture: amd64
Connected to source registry: index.docker.io
Connected to destination registry: localhost:5555
Using base image: index.docker.io/library/swift:slim
Found base image manifest: sha256:bcbe7e057d0a9a9685bce1917efb8fd3fe350a28e6f1b8184989553933e1bc96
Found base image configuration: sha256:8dae9dd90c7bc6eefac511b3fe38e2238741fcd90327640cd4a0397139cc1773
Error: dataCorrupted(Swift.DecodingError.Context(codingPath: [], debugDescription: "The given data was not valid JSON.", underlyingError: Optional(Error Domain=NSCocoaErrorDomain Code=3840 "Unexpected character 'p' after top-level value around line 1, column 5." UserInfo={NSJSONSerializationErrorIndex=4, NSDebugDescription=Unexpected character 'p' after top-level value around line 1, column 5.})))

[euanh]

Thank you for reporting this, @dirk68-fu.

I'm afraid I can't currently reproduce the problem you are seeing. If I try the same command, containertool is able to pull swift:slim and push the completed image back up to the local registry.

% swift run containertool .build/x86_64-swift-linux-musl/debug/hello-world --repository localhost:5555/hello -v --disable-netrc
Building for debugging...
[1/1] Write swift-version-1AF32027EA501FB9.txt
Build of product 'containertool' complete! (0.67s)
Base image architecture: amd64
Connected to source registry: index.docker.io
Connected to destination registry: localhost:5555
Using base image: index.docker.io/library/swift:slim
Found base image manifest: sha256:bcbe7e057d0a9a9685bce1917efb8fd3fe350a28e6f1b8184989553933e1bc96
Found base image configuration: sha256:8dae9dd90c7bc6eefac511b3fe38e2238741fcd90327640cd4a0397139cc1773

<<< your run fails here >>>

application layer: sha256:cdd50a07e8829cdcda1c57a95d6b975acfc9d61730cfd9abf8a1ef960e95aa1b (66382985 bytes)
image configuration: sha256:595f2a02d77f438f697fbf48f5e657ead0142cb8feb6cc4dd486a30b0770cca1 (1340 bytes)
Layer sha256:0622fac788edde5d30e7bbd2688893e5452a19ff237a2e4615e2d8181321cb4e: fetching
Layer sha256:0622fac788edde5d30e7bbd2688893e5452a19ff237a2e4615e2d8181321cb4e: pushing
Layer sha256:0622fac788edde5d30e7bbd2688893e5452a19ff237a2e4615e2d8181321cb4e: done
Layer sha256:bce0b04dd670dff2c391089e5d1fb8542f8bfe9b7d708b7dd91a439305e4875a: fetching
Layer sha256:bce0b04dd670dff2c391089e5d1fb8542f8bfe9b7d708b7dd91a439305e4875a: pushing
Layer sha256:bce0b04dd670dff2c391089e5d1fb8542f8bfe9b7d708b7dd91a439305e4875a: done
Layer sha256:1e40fa52d9559322924564170c65c2fceaed3e9c40974b7d5d81a49c6409c3d9: fetching
Layer sha256:1e40fa52d9559322924564170c65c2fceaed3e9c40974b7d5d81a49c6409c3d9: pushing
Layer sha256:1e40fa52d9559322924564170c65c2fceaed3e9c40974b7d5d81a49c6409c3d9: done
http://localhost:5555/v2/hello/manifests/sha256:4c7d791a56b3dc3982c4117fd6cca18737e07df65ff80a0ea4d1a4c7a6869481
localhost:5555/hello@sha256:4c7d791a56b3dc3982c4117fd6cca18737e07df65ff80a0ea4d1a4c7a6869481

The Connected to source registry and Connected to destination registry lines show that containertool connected to both registries and checked that they both speak the registry protocol.

In your run, containertool found the same base image manifest and configuration as mine. These are both JSON structures. Since my run was able to parse these structures, we should be safe to assume that your run received the same JSON and was also able to parse it.

After identifying the base image, containertool tries to upload resource bundles. Since you are not passing the --resources flag that step is skipped so we can rule out problems here.

The next step is then building and uploading the application layer. The application layer: line is printed after the upload finishes, so it looks like your run encountered a problem in this step. A successful blob upload doesn't return a JSON object, but if the registry returns an error it may contain a JSON payload. It looks like in your run the registry may have returned an error and containertool failed to parse it.

Could you please provide a bit more information to help me investigate further?

• What platform are you running on? Please post the output of the following commands:

  sw_vers              # only if you are running on macOS
  echo $TOOLCHAINS     # only on macOS
  swift --version
  

• Since the registry may be returning an error, please post its logs during the failed upload:

  docker logs <ID of registry container>
  

[dirk68-fu]

The problem seems to occur if you have an uppercase letter in the part after

localhost:5555/

localhost:5555/hello works but localhost:5555/helloServer does not.

Is this a restriction of the registry or a bug in swift-container-plugin?
Better error message would certainly be helpful ;-)

[euanh]

The problem seems to occur if you have an uppercase letter...

Yes, I can reproduce this problem with helloWorld instead of hello-world. Well spotted!

Is this a restriction of the registry or a bug in swift-container-plugin?

It's a restriction of the registry.

The root cause is that capital letters are not allowed in the repository name. If you try to use the docker CLI to give an image a tag which contains a capital, you get an error. The podman CLI has does the same.

% docker image tag localhost:5555/hello-world:latest localhost:5555/helloWorld:latest
Error parsing reference: "localhost:5555/helloWorld:latest" is not a valid repository/tag: invalid reference format: repository name (helloWorld) must be lowercase

% docker image tag localhost:5555/hello-world:latest helloWorld       
Error parsing reference: "helloWorld" is not a valid repository/tag: invalid reference format: repository name (library/helloWorld) must be lowercase

The allowable characters in a repository name are defined here in the distribution spec.

Better error message would certainly be helpful ;-)

Yes, definitely.

The error message you see is printed because containertool tries to upload an image with a capital in the name, the registry response is just a 404 with a plain string body of page not found. containertool tries to parse this as a JSON error and fails - this is why the error message is Unexpected character 'p' after top-level value around line 1, column 5..

There are two problems with containertool:

• containertool should reject capital letters in repository names #135
• ContainerRegistry should handle non-JSON error payloads more gracefully #136

In the meantime you can work around this problem by avoiding capital letters in the --repository argument.

[euanh]

The documentation and the conference videos don't show how to run a local registry which swift-container-plugin needs.

Just for completeness, swift-container-plugin doesn't require the use of a local registry. You can push your image to a public registry such as Docker Hub, ECR or GHCR, or a private registry running somewhere on the network, maybe provided by your company. You will need an account to push to such a registry, and the documentation explains how to set up credentials so containertool / swift-container-plugin can authenticate.

If those options aren't suitable, you can run a local registry instance. The demos and presentations tend to use the local registry approach because:

• not everyone will have an account at any particular public registry
• wifi at conferences can be unreliable, so pushing to a remote registry during a demo is not practical

Being able to send an image directly to the local container runtime, or save it to a disk image, would give another option. However as discussed it's not currently supported.

The documentation should explain this better: #137