Podman with Nexus proxy - unsupported schema version 2 #14447
Replies: 3 comments 7 replies
-
|
@vrothberg PTAL |
Beta Was this translation helpful? Give feedback.
-
|
This is the output I get with --log-level=debug: The Nexus proxy repository contains two files after this: The content of the files are:
{
"manifests": [
{
"digest": "sha256:f54a58bc1aac5ea1a25d796ae155dc228b3f0e11d046ae276b39c4bf2f13d8c4",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "amd64",
"os": "linux"
},
"size": 525
},
{
"digest": "sha256:7b8b7289d0536a08eabdf71c20246e23f7116641db7e1d278592236ea4dcb30c",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "arm",
"os": "linux",
"variant": "v5"
},
"size": 525
},
{
"digest": "sha256:f130bd2d67e6e9280ac6d0a6c83857bfaf70234e8ef4236876eccfbd30973b1c",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "arm",
"os": "linux",
"variant": "v7"
},
"size": 525
},
{
"digest": "sha256:432f982638b3aefab73cc58ab28f5c16e96fdb504e8c134fc58dff4bae8bf338",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "arm64",
"os": "linux",
"variant": "v8"
},
"size": 525
},
{
"digest": "sha256:995efde2e81b21d1ea7066aa77a59298a62a9e9fbb4b77f36c189774ec9b1089",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "386",
"os": "linux"
},
"size": 525
},
{
"digest": "sha256:eb11b1a194ff8e236a01eff392c4e1296a53b0fb4780d8b0382f7996a15d5392",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "mips64le",
"os": "linux"
},
"size": 525
},
{
"digest": "sha256:b836bb24a270b9cc935962d8228517fde0f16990e88893d935efcb1b14c0017a",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "ppc64le",
"os": "linux"
},
"size": 525
},
{
"digest": "sha256:98c9722322be649df94780d3fbe594fce7996234b259f27eac9428b84050c849",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "riscv64",
"os": "linux"
},
"size": 525
},
{
"digest": "sha256:c7b6944911848ce39b44ed660d95fb54d69bbd531de724c7ce6fc9f743c0b861",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "s390x",
"os": "linux"
},
"size": 525
},
{
"digest": "sha256:1e0054d86f2721056c9db5c2322fb8883d185fa1165bca69cb0ac88eeb40be30",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "amd64",
"os": "windows",
"os.version": "10.0.20348.707"
},
"size": 1125
},
{
"digest": "sha256:02a91a3365149788cc22339941adefe48166286642688a1f094c3517da4e37a5",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "amd64",
"os": "windows",
"os.version": "10.0.17763.2928"
},
"size": 1125
}
],
"mediaType": "application\/vnd.docker.distribution.manifest.list.v2+json",
"schemaVersion": 2
}
{
"manifests": [
{
"digest": "sha256:f54a58bc1aac5ea1a25d796ae155dc228b3f0e11d046ae276b39c4bf2f13d8c4",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "amd64",
"os": "linux"
},
"size": 525
},
{
"digest": "sha256:7b8b7289d0536a08eabdf71c20246e23f7116641db7e1d278592236ea4dcb30c",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "arm",
"os": "linux",
"variant": "v5"
},
"size": 525
},
{
"digest": "sha256:f130bd2d67e6e9280ac6d0a6c83857bfaf70234e8ef4236876eccfbd30973b1c",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "arm",
"os": "linux",
"variant": "v7"
},
"size": 525
},
{
"digest": "sha256:432f982638b3aefab73cc58ab28f5c16e96fdb504e8c134fc58dff4bae8bf338",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "arm64",
"os": "linux",
"variant": "v8"
},
"size": 525
},
{
"digest": "sha256:995efde2e81b21d1ea7066aa77a59298a62a9e9fbb4b77f36c189774ec9b1089",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "386",
"os": "linux"
},
"size": 525
},
{
"digest": "sha256:eb11b1a194ff8e236a01eff392c4e1296a53b0fb4780d8b0382f7996a15d5392",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "mips64le",
"os": "linux"
},
"size": 525
},
{
"digest": "sha256:b836bb24a270b9cc935962d8228517fde0f16990e88893d935efcb1b14c0017a",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "ppc64le",
"os": "linux"
},
"size": 525
},
{
"digest": "sha256:98c9722322be649df94780d3fbe594fce7996234b259f27eac9428b84050c849",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "riscv64",
"os": "linux"
},
"size": 525
},
{
"digest": "sha256:c7b6944911848ce39b44ed660d95fb54d69bbd531de724c7ce6fc9f743c0b861",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "s390x",
"os": "linux"
},
"size": 525
},
{
"digest": "sha256:1e0054d86f2721056c9db5c2322fb8883d185fa1165bca69cb0ac88eeb40be30",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "amd64",
"os": "windows",
"os.version": "10.0.20348.707"
},
"size": 1125
},
{
"digest": "sha256:02a91a3365149788cc22339941adefe48166286642688a1f094c3517da4e37a5",
"mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
"platform": {
"architecture": "amd64",
"os": "windows",
"os.version": "10.0.17763.2928"
},
"size": 1125
}
],
"mediaType": "application\/vnd.docker.distribution.manifest.list.v2+json",
"schemaVersion": 2
}Is there anything else I can look into locally for finding more information about the problem at hand? |
Beta Was this translation helpful? Give feedback.
-
The proxy, or maybe the original registry is providing an invalid value here. It must be fixed to return the right value (this is specific to each individual manifest, it can’t be a simple “everything with JSON suffix, or everything in a /manifests/ URL, should report a single value). |
Beta Was this translation helpful? Give feedback.
-
|
the “invalid value here” refers to a |
Beta Was this translation helpful? Give feedback.
-
|
I'll look into the configuration of the proxy repository in the Nexus server. There are not that much to configure though but I guess it could be a bug in Nexus. |
Beta Was this translation helpful? Give feedback.
-
|
It is possible to workaround this problem with the following procedure:
I'm not sure that this is a problem with podman, but it would be good to continue the investigation further to find the root cause to the problem. |
Beta Was this translation helpful? Give feedback.
-
|
@mtrmac PTAL |
Beta Was this translation helpful? Give feedback.
-
|
That workaround seems unlikely. Are steps 1. and 3. both done from the same computer? If so, does that work if the local image store is completely purged in between? How do the two images actually differ (maybe using |
Beta Was this translation helpful? Give feedback.
-
|
All three steps are performed on the same computer. The local image store is completely purged between step 2 and 3. I do not have access to skopeo in the environment so I cannot inspect the images. What I can see in Nexus is however that the manifest and tag files differs. I've attached the non proxy file contents below. They are completely different from the files downloaded when pulling the image through the Nexus proxy repository. latest sha256... I'm going on vacation for 4 weeks so I will not be able to respond until 25th of July. |
Beta Was this translation helpful? Give feedback.
-
|
That… mostly seems to just introduce extra variables into the investigation. The core of the issue is that with invalid Content-Type, the code defaults to the oldest schema1 ("schemaVersion": "1"). Something in the workaround process, for a probably fairly independent reason, triggers conversion of the original multi-arch image (80f31da1ac7b312ba29d65080fddf797dd76acfb870e677f390d5acba9741b17) to schema1; probably pushes using other versions are rejected, for some reason, and the push code fall backs to schema1. (Or, alternatively, the proxy is so badly misconfigured, that it triggers a server-side conversion to schema1.) Either way, now that the freshly-pushed image really is schema1, it probably matches with the invalid Content-type header, and can be pulled successfully. (Also note that the workaround, using |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
I'm using podman 4.0.2 in an offgrid environment with a Nexus proxy behind an Artifactory. The Artifactory proxy is the only way to retrieve images from. Artifactory has no long term storage guaranteed for the images so I need the Nexus to store the downloaded images long term.
When I try to pull images I get the following error when going through the Nexus proxy, but not when only going through the Artifactory proxy:
Nexus contains json files for each image downloaded, latests and a sha256:xxxxx. The content in each of the json files contains:
"mediaType": "application\/vnd.docker.distribution.manifest.list.v2+json"Can anyone give me some hints on how to fix this or further investigate why I get this error?
Beta Was this translation helpful? Give feedback.
All reactions