docker bake push fails with manifest invalid: manifest invalid for another user but works for me

[ieugen]

Hello,

I've setup zot for evaluation and we have a mixed results.
For me it works, for him it failed when he tried to push just now.
We use a docker bake file to publish the image - he might have used docker build as well.

He is on docker 28.1 and I am on docker 29.0.3 .

Any idea what is happening ?
The error message from zot is not very helpful.

We'll try to upgrade his docker version, but i could not find anything related to this in the docker release notes.
Will check as well the release notes for buildx .

[andaaron]

Hi @ieugen, can you share the server side logs? What media types are you using for images, maybe you are missing https://zotregistry.dev/v2.1.11/admin-guide/admin-configuration/#compatibility-with-other-image-schema-types?

[ieugen]

this is what I get inspecting the image (push works for me)

docker buildx imagetools inspect REDACTED/flowintel/flow-intel-v2-server:dev-latest
Name:      REDACTED/flowintel/flow-intel-v2-server:dev-latest
MediaType: application/vnd.oci.image.index.v1+json
Digest:    sha256:acf5bb00c09e6c9d0042f5d1936bf614d32e2f194a458da4683e14582613d5fd

Manifests:
  Name:        REDACTED/flowintel/flow-intel-v2-server:dev-latest@sha256:6348a9b9fed5bfeae5337590a40e02d0cc1e9b1d4daee9f79661dad32a3a48da
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    linux/amd64

  Name:        REDACTED/flowintel/flow-intel-v2-server:dev-latest@sha256:b91a242863d5bf0e6d7c7a80c6bec00c5569471efe409595b674c893c359acbb
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    unknown/unknown
  Annotations:
    vnd.docker.reference.digest: sha256:6348a9b9fed5bfeae5337590a40e02d0cc1e9b1d4daee9f79661dad32a3a48da
    vnd.docker.reference.type:   attestation-manifest

[ieugen]

I also have this option enabled, not sure if it matters.

$ cat /etc/docker/daemon.json
{
  "features": {
    "containerd-snapshotter": true
  }
}

[andaaron]

this is what I get inspecting the image (push works for me)

docker buildx imagetools inspect REDACTED/flowintel/flow-intel-v2-server:dev-latest
Name:      REDACTED/flowintel/flow-intel-v2-server:dev-latest
MediaType: application/vnd.oci.image.index.v1+json
Digest:    sha256:acf5bb00c09e6c9d0042f5d1936bf614d32e2f194a458da4683e14582613d5fd

Manifests:
  Name:        REDACTED/flowintel/flow-intel-v2-server:dev-latest@sha256:6348a9b9fed5bfeae5337590a40e02d0cc1e9b1d4daee9f79661dad32a3a48da
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    linux/amd64

  Name:        REDACTED/flowintel/flow-intel-v2-server:dev-latest@sha256:b91a242863d5bf0e6d7c7a80c6bec00c5569471efe409595b674c893c359acbb
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    unknown/unknown
  Annotations:
    vnd.docker.reference.digest: sha256:6348a9b9fed5bfeae5337590a40e02d0cc1e9b1d4daee9f79661dad32a3a48da
    vnd.docker.reference.type:   attestation-manifest

Can you provide the same info for the image which doesn't work?
L.E. Actually it is not needed in the logs I see "Content-Type":["application/vnd.docker.distribution.manifest.v2+json"]

[andaaron]

Do you have this in your zot config "compat": ["docker2s2"]? Under http.

Per https://zotregistry.dev/v2.1.11/admin-guide/admin-configuration/#compatibility-with-other-image-schema-types

[ieugen]

No. Thanks, I read the config pages but probably missed this one.
I added and deployed, but dev with push issue is on PTO for a week.
I left him a message to try it out when he's back.

Will come back with info when I have it.

[ieugen]

A side note, any idea why I see: Platform: unknown/unknown ?
Seems awkward and raises questions from users - to which I don't have any answers :( .
I've found an explanation of why these appear in general (github) - any open issues for it in zot?

https://github.com/orgs/community/discussions/45969

[andaaron]

That manifest with os/arch unknown/unknown does not represent an image which could be run on a specific operating system/architecture. It is a Docker build atestation, which serves a different purpose. That 'platform' doesn't make sense in the context of that manifest so it's populates with some filler values unknown

[ieugen]

docker bake push now works after using "compat": ["docker2s2"]