docker : publish to both ggerganov and ggml-org

[ngxson]

Ref discussion: #11801 (comment)

For now I don't know how to test it, but 🤞 hopefully it works.

You need to make sure that Manage Actions access allows the correct repo to access the registry package, please check in:

• https://github.com/users/ggml-org/packages/container/llama.cpp/settings
• https://github.com/users/ggerganov/packages/container/llama.cpp/settings

Here is what I see under my fork at https://github.com/ngxson/llama.cpp

[ggerganov]

The ggerganov package cannot link to the ggml-org repos:

And the ggml-org package is currently not existing. I am not sure how to create it yet:

[ngxson]

The ggerganov package cannot link to the ggml-org repos

Hmm that's a bummer, github poorly documented this part.

Another way though, we can create another repo under your personal account, let's say ggerganov/llama.cpp-ci-mirror that have access to ggerganov/llama.cpp package. Then setup a cronjob to mirror the package from ggml-org back to your personal account.

And the ggml-org package is currently not existing. I am not sure how to create it yet

I think you need to run the CI at least once, the registry will be created whenever a new image is pushed.

[ngxson]

I triggered the docker CI: https://github.com/ggml-org/llama.cpp/actions/runs/13345981032

It will create ggml-org/llama.cpp

[ggerganov]

Yes, I think the mirror will work - will set it up later.

Do we still need this PR in this case?

[ngxson]

Do we still need this PR in this case?

No I'll close this, but we can keep discussing about the mirror here if needed.

[ggerganov]

This one failed because I forgot to enable write permissions for the Github Actions workflows:

https://github.com/ggml-org/llama.cpp/actions/runs/13345981032/job/37276456430#step:8:1129

The next pushes should be good.

[ggerganov]

I think we are gucci: https://github.com/ggml-org/llama.cpp/pkgs/container/llama.cpp

[ggerganov]

Regarding the mirror repo - I wonder if we really need to do it. It will just keep people stuck to the ggerganov package since it is being updated anyway.

I am thinking about some alternative - maybe better to put a dummy repo with a README that the package has moved.

[ngxson]

Regarding the mirror repo - I wonder if we really need to do it. It will just keep people stuck to the ggerganov package since it is being updated anyway.

This is mostly for less technical people just copy-paste docker run command from the internet. They may not pay attention to this change, so a mirror is still needed. I would say this group of people make up a large part of llama.cpp's user base, so I don't want to break their workflow.

I am thinking about some alternative - maybe better to put a dummy repo with a README that the package has moved.

This will permanently break the github redirection so I think it's quite destructive. IMO we should always keep that redirection, in case someone already had automated scripts that depends on github.com/ggerganov/llama.cpp

That saying, I will experiment with a mirror on my side. My plan is to include a script that get the diff between 2 registries and only sync what is needed (one-way sync from ggml-org to ggerganov). It should be straight-forward to do, I'll share when it's ready.

[ggerganov]

IMO we should always keep that redirection, in case someone already had automated scripts that depends on github.com/ggerganov/llama.cpp

Just to clarify, my idea is to create ggerganov/llama.cpp-dummy that will only have a README file stating the package has moved. And then link that dummy repo to https://github.com/users/ggerganov/packages/container/package/llama.cpp so that anyone who visits the package page would know that it has moved. The drawback of course is that package will remain stuck at version b4721 and users might not realize that they are not using the latest version.

That saying, I will experiment with a mirror on my side. My plan is to include a script that get the diff between 2 registries and only sync what is needed (one-way sync from ggml-org to ggerganov). It should be straight-forward to do, I'll share when it's ready.

Ok, let me know if you come up with something.

[ngxson]

@ggerganov Here is the code to mirror docker image between 2 repos: https://github.com/ngxson/llama.cpp-publish-mirror

It fails for now because there is no CPU tag on this ggml-org repo (the CI failed due to that segfault error on ARM64), but it should work, I tested with the mirror from ggerganov to my repo: https://github.com/ngxson/llama.cpp-publish-mirror/actions/runs/13348775563/job/37282572268

The speed is very fast because it only needs to push the manifest, no need to download/re-upload the actual data.

Let me know if you need more help to set it up.

[ngxson]

The CPU build is fixed, and I was able to run the mirror script successfully. It only takes 35s to finish: https://github.com/ngxson/llama.cpp-publish-mirror/actions/runs/13354165350/job/37294577156

And here is the output (a package linked to another project): https://github.com/ngxson/llama.cpp-test-mirror/pkgs/container/llama.cpp-test-mirror

[ggerganov]

I forked the mirror here: https://github.com/ggerganov/llama.cpp-publish-mirror

I noticed that the synced containers don't have a release tag:

Not sure if this is expected.

[ggerganov]

I think the HF endpoints are failing now when using the default ghcr.io/ggerganov/llama.cpp:server-cuda. However there is no log available, so I'm not sure what is the problem.

[ggerganov]

I think the HF endpoints are failing now when using the default ghcr.io/ggerganov/llama.cpp:server-cuda. However there is no log available, so I'm not sure what is the problem.

Fixed via ggerganov/llama.cpp-publish-mirror@c7cb45d