- 
                Notifications
    You must be signed in to change notification settings 
- Fork 13.5k
devops: add s390x containers #15915
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
devops: add s390x containers #15915
Conversation
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
failure to load model Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
This reverts commit 0a7664a. Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]> (cherry picked from commit 0a7664a) Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
| I notice that my patched OpenBLAS has slower performance for Prompt Processing but faster performance for Token Generation. This doesn't look right, I might revert back to the distributed OpenBLAS instead. | 
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
| Reverted back to using  
 Also, I've added s390x docker build to  Edit: My custom OpenBLAS patch is way faster without Docker. But I can't seem to replicate the same performance oh well. 
 | 
* origin/master: (39 commits) ci : disable AMD workflows + update NVIDIA workflows (ggml-org#16200) ci : enable Vulkan workflow on Mac (ggml-org#16194) ggml-cpu: Respect cpumask settings (ggml-org#16164) ggml : fix uninitialized is_on_grid in quantize_row_iq3_xxs_impl (ggml-org#15928) zdnn: refactor codebase + add docs (ggml-org#16178) codeowners : add @danbev to model-conversion example [no ci] (ggml-org#16190) devops: add s390x containers (ggml-org#15915) ggml-cpu : fix typo in gemm comments [no ci] (ggml-org#16189) feat: Add conversion support in GraniteHybrid for non-hybrid (all attn) (ggml-org#16177) clang-tidy : disable warning about performance enum size (ggml-org#16127) ggml : implement set_rows with i32 index (ggml-org#16159) codeowners : update + cleanup (ggml-org#16174) common : enable `--offline` mode without curl support (ggml-org#16137) webui : fix handling incomplete chunks (ggml-org#16107) embedding : fix typos in README (ggml-org#16171) common : remove unused local variables (ggml-org#16140) ggml : extend ggml_can_fuse to work with non-sequential nodes (ggml-org#16123) ggml : add ggml_op_is_empty (ggml-org#16122) codeowners : update ownership for @ngxson and @allozuar (ggml-org#16128) Vulkan: add conv_transpose_2d operation (ggml-org#16022) ...
* devops: add s390x dockerfile Signed-off-by: Aaron Teo <[email protected]> * devops: add missing ninja Signed-off-by: Aaron Teo <[email protected]> * devops: move s390x docker into cpu docker Signed-off-by: Aaron Teo <[email protected]> * devops: rework s390x docker Signed-off-by: Aaron Teo <[email protected]> * devops: copy more tools Signed-off-by: Aaron Teo <[email protected]> * devops: add server build step Signed-off-by: Aaron Teo <[email protected]> * devops: remove apt clean steps as distroless misses it Signed-off-by: Aaron Teo <[email protected]> * devops: remove apt commands from distroless Signed-off-by: Aaron Teo <[email protected]> * devops: fix shared libs in distroless Signed-off-by: Aaron Teo <[email protected]> * devops: use correct libs path Signed-off-by: Aaron Teo <[email protected]> * devops: fix shared libs Signed-off-by: Aaron Teo <[email protected]> * devops: add collector stage Signed-off-by: Aaron Teo <[email protected]> * devops: fix missing stage ref Signed-off-by: Aaron Teo <[email protected]> * devops: fix permission issue Signed-off-by: Aaron Teo <[email protected]> * devops: fix unknown model loading failures Signed-off-by: Aaron Teo <[email protected]> * devops: attempt at fixing model loading failure Signed-off-by: Aaron Teo <[email protected]> * devops: fix missing ggml shared object failure to load model Signed-off-by: Aaron Teo <[email protected]> * devops: remove move shared objects Signed-off-by: Aaron Teo <[email protected]> * devops: move libggml-cpu and blas into bin Signed-off-by: Aaron Teo <[email protected]> * devops: finalise hardened server stage Signed-off-by: Aaron Teo <[email protected]> * devops: add cli target Signed-off-by: Aaron Teo <[email protected]> * devops: fix typos Signed-off-by: Aaron Teo <[email protected]> * devops: fix missing shared libraries in base Signed-off-by: Aaron Teo <[email protected]> * devops: update debian target Signed-off-by: Aaron Teo <[email protected]> * devops: formalise llama.cpp loc Signed-off-by: Aaron Teo <[email protected]> * Revert "devops: formalise llama.cpp loc" This reverts commit 0a7664a. Signed-off-by: Aaron Teo <[email protected]> * devops: formalise llama.cpp loc Signed-off-by: Aaron Teo <[email protected]> (cherry picked from commit 0a7664a) Signed-off-by: Aaron Teo <[email protected]> * devops: attempt at fixing missing dir Signed-off-by: Aaron Teo <[email protected]> * devops: attempt at making it cache the build Signed-off-by: Aaron Teo <[email protected]> * devops: fix copying process Signed-off-by: Aaron Teo <[email protected]> * devops: make build dir an argument Signed-off-by: Aaron Teo <[email protected]> * Revert "devops: make build dir an argument" This reverts commit 4386989. Signed-off-by: Aaron Teo <[email protected]> * devops: add build stage for gguf-py Signed-off-by: Aaron Teo <[email protected]> * devops: move gguf-py installation into build stage Signed-off-by: Aaron Teo <[email protected]> * devops: break system packages? Signed-off-by: Aaron Teo <[email protected]> * devops: add rust compiler installer Signed-off-by: Aaron Teo <[email protected]> * devops: fix rustc not found Signed-off-by: Aaron Teo <[email protected]> * devops: remove cache mount to allow rustc to persist Signed-off-by: Aaron Teo <[email protected]> * devops: move rustc installation to another layer Signed-off-by: Aaron Teo <[email protected]> * devops: move gguf-py installation to full stage, fix copying Signed-off-by: Aaron Teo <[email protected]> * devops: remove rustc installation in build Signed-off-by: Aaron Teo <[email protected]> * devops: disable full target for now Signed-off-by: Aaron Teo <[email protected]> * devops: attempting static build Signed-off-by: Aaron Teo <[email protected]> * devops: merge s390x dockerfile into cpu for now Signed-off-by: Aaron Teo <[email protected]> * devops: switch to gcc image for build step Signed-off-by: Aaron Teo <[email protected]> * devops: remove build essentials Signed-off-by: Aaron Teo <[email protected]> * devops: install openblas into base target Signed-off-by: Aaron Teo <[email protected]> * devops: go back to s390x dockerfile Signed-off-by: Aaron Teo <[email protected]> * devops: remove libggml and libblas Signed-off-by: Aaron Teo <[email protected]> * devops: add full target Signed-off-by: Aaron Teo <[email protected]> * devops: add break system packages Signed-off-by: Aaron Teo <[email protected]> * devops: add libjpeg Signed-off-by: Aaron Teo <[email protected]> * devops: add missing cmake dep Signed-off-by: Aaron Teo <[email protected]> * devops: finalise docker images for s390x Signed-off-by: Aaron Teo <[email protected]> * devops: add custom openblas patch Signed-off-by: Aaron Teo <[email protected]> * devops: use libopenblas-dev instead of libopenblas-openmp-dev Signed-off-by: Aaron Teo <[email protected]> * devops: add s390x docker build Signed-off-by: Aaron Teo <[email protected]> --------- Signed-off-by: Aaron Teo <[email protected]>
* devops: add s390x dockerfile Signed-off-by: Aaron Teo <[email protected]> * devops: add missing ninja Signed-off-by: Aaron Teo <[email protected]> * devops: move s390x docker into cpu docker Signed-off-by: Aaron Teo <[email protected]> * devops: rework s390x docker Signed-off-by: Aaron Teo <[email protected]> * devops: copy more tools Signed-off-by: Aaron Teo <[email protected]> * devops: add server build step Signed-off-by: Aaron Teo <[email protected]> * devops: remove apt clean steps as distroless misses it Signed-off-by: Aaron Teo <[email protected]> * devops: remove apt commands from distroless Signed-off-by: Aaron Teo <[email protected]> * devops: fix shared libs in distroless Signed-off-by: Aaron Teo <[email protected]> * devops: use correct libs path Signed-off-by: Aaron Teo <[email protected]> * devops: fix shared libs Signed-off-by: Aaron Teo <[email protected]> * devops: add collector stage Signed-off-by: Aaron Teo <[email protected]> * devops: fix missing stage ref Signed-off-by: Aaron Teo <[email protected]> * devops: fix permission issue Signed-off-by: Aaron Teo <[email protected]> * devops: fix unknown model loading failures Signed-off-by: Aaron Teo <[email protected]> * devops: attempt at fixing model loading failure Signed-off-by: Aaron Teo <[email protected]> * devops: fix missing ggml shared object failure to load model Signed-off-by: Aaron Teo <[email protected]> * devops: remove move shared objects Signed-off-by: Aaron Teo <[email protected]> * devops: move libggml-cpu and blas into bin Signed-off-by: Aaron Teo <[email protected]> * devops: finalise hardened server stage Signed-off-by: Aaron Teo <[email protected]> * devops: add cli target Signed-off-by: Aaron Teo <[email protected]> * devops: fix typos Signed-off-by: Aaron Teo <[email protected]> * devops: fix missing shared libraries in base Signed-off-by: Aaron Teo <[email protected]> * devops: update debian target Signed-off-by: Aaron Teo <[email protected]> * devops: formalise llama.cpp loc Signed-off-by: Aaron Teo <[email protected]> * Revert "devops: formalise llama.cpp loc" This reverts commit 0a7664a. Signed-off-by: Aaron Teo <[email protected]> * devops: formalise llama.cpp loc Signed-off-by: Aaron Teo <[email protected]> (cherry picked from commit 0a7664a) Signed-off-by: Aaron Teo <[email protected]> * devops: attempt at fixing missing dir Signed-off-by: Aaron Teo <[email protected]> * devops: attempt at making it cache the build Signed-off-by: Aaron Teo <[email protected]> * devops: fix copying process Signed-off-by: Aaron Teo <[email protected]> * devops: make build dir an argument Signed-off-by: Aaron Teo <[email protected]> * Revert "devops: make build dir an argument" This reverts commit 4386989. Signed-off-by: Aaron Teo <[email protected]> * devops: add build stage for gguf-py Signed-off-by: Aaron Teo <[email protected]> * devops: move gguf-py installation into build stage Signed-off-by: Aaron Teo <[email protected]> * devops: break system packages? Signed-off-by: Aaron Teo <[email protected]> * devops: add rust compiler installer Signed-off-by: Aaron Teo <[email protected]> * devops: fix rustc not found Signed-off-by: Aaron Teo <[email protected]> * devops: remove cache mount to allow rustc to persist Signed-off-by: Aaron Teo <[email protected]> * devops: move rustc installation to another layer Signed-off-by: Aaron Teo <[email protected]> * devops: move gguf-py installation to full stage, fix copying Signed-off-by: Aaron Teo <[email protected]> * devops: remove rustc installation in build Signed-off-by: Aaron Teo <[email protected]> * devops: disable full target for now Signed-off-by: Aaron Teo <[email protected]> * devops: attempting static build Signed-off-by: Aaron Teo <[email protected]> * devops: merge s390x dockerfile into cpu for now Signed-off-by: Aaron Teo <[email protected]> * devops: switch to gcc image for build step Signed-off-by: Aaron Teo <[email protected]> * devops: remove build essentials Signed-off-by: Aaron Teo <[email protected]> * devops: install openblas into base target Signed-off-by: Aaron Teo <[email protected]> * devops: go back to s390x dockerfile Signed-off-by: Aaron Teo <[email protected]> * devops: remove libggml and libblas Signed-off-by: Aaron Teo <[email protected]> * devops: add full target Signed-off-by: Aaron Teo <[email protected]> * devops: add break system packages Signed-off-by: Aaron Teo <[email protected]> * devops: add libjpeg Signed-off-by: Aaron Teo <[email protected]> * devops: add missing cmake dep Signed-off-by: Aaron Teo <[email protected]> * devops: finalise docker images for s390x Signed-off-by: Aaron Teo <[email protected]> * devops: add custom openblas patch Signed-off-by: Aaron Teo <[email protected]> * devops: use libopenblas-dev instead of libopenblas-openmp-dev Signed-off-by: Aaron Teo <[email protected]> * devops: add s390x docker build Signed-off-by: Aaron Teo <[email protected]> --------- Signed-off-by: Aaron Teo <[email protected]>
* devops: add s390x dockerfile Signed-off-by: Aaron Teo <[email protected]> * devops: add missing ninja Signed-off-by: Aaron Teo <[email protected]> * devops: move s390x docker into cpu docker Signed-off-by: Aaron Teo <[email protected]> * devops: rework s390x docker Signed-off-by: Aaron Teo <[email protected]> * devops: copy more tools Signed-off-by: Aaron Teo <[email protected]> * devops: add server build step Signed-off-by: Aaron Teo <[email protected]> * devops: remove apt clean steps as distroless misses it Signed-off-by: Aaron Teo <[email protected]> * devops: remove apt commands from distroless Signed-off-by: Aaron Teo <[email protected]> * devops: fix shared libs in distroless Signed-off-by: Aaron Teo <[email protected]> * devops: use correct libs path Signed-off-by: Aaron Teo <[email protected]> * devops: fix shared libs Signed-off-by: Aaron Teo <[email protected]> * devops: add collector stage Signed-off-by: Aaron Teo <[email protected]> * devops: fix missing stage ref Signed-off-by: Aaron Teo <[email protected]> * devops: fix permission issue Signed-off-by: Aaron Teo <[email protected]> * devops: fix unknown model loading failures Signed-off-by: Aaron Teo <[email protected]> * devops: attempt at fixing model loading failure Signed-off-by: Aaron Teo <[email protected]> * devops: fix missing ggml shared object failure to load model Signed-off-by: Aaron Teo <[email protected]> * devops: remove move shared objects Signed-off-by: Aaron Teo <[email protected]> * devops: move libggml-cpu and blas into bin Signed-off-by: Aaron Teo <[email protected]> * devops: finalise hardened server stage Signed-off-by: Aaron Teo <[email protected]> * devops: add cli target Signed-off-by: Aaron Teo <[email protected]> * devops: fix typos Signed-off-by: Aaron Teo <[email protected]> * devops: fix missing shared libraries in base Signed-off-by: Aaron Teo <[email protected]> * devops: update debian target Signed-off-by: Aaron Teo <[email protected]> * devops: formalise llama.cpp loc Signed-off-by: Aaron Teo <[email protected]> * Revert "devops: formalise llama.cpp loc" This reverts commit 0a7664a. Signed-off-by: Aaron Teo <[email protected]> * devops: formalise llama.cpp loc Signed-off-by: Aaron Teo <[email protected]> (cherry picked from commit 0a7664a) Signed-off-by: Aaron Teo <[email protected]> * devops: attempt at fixing missing dir Signed-off-by: Aaron Teo <[email protected]> * devops: attempt at making it cache the build Signed-off-by: Aaron Teo <[email protected]> * devops: fix copying process Signed-off-by: Aaron Teo <[email protected]> * devops: make build dir an argument Signed-off-by: Aaron Teo <[email protected]> * Revert "devops: make build dir an argument" This reverts commit 4386989. Signed-off-by: Aaron Teo <[email protected]> * devops: add build stage for gguf-py Signed-off-by: Aaron Teo <[email protected]> * devops: move gguf-py installation into build stage Signed-off-by: Aaron Teo <[email protected]> * devops: break system packages? Signed-off-by: Aaron Teo <[email protected]> * devops: add rust compiler installer Signed-off-by: Aaron Teo <[email protected]> * devops: fix rustc not found Signed-off-by: Aaron Teo <[email protected]> * devops: remove cache mount to allow rustc to persist Signed-off-by: Aaron Teo <[email protected]> * devops: move rustc installation to another layer Signed-off-by: Aaron Teo <[email protected]> * devops: move gguf-py installation to full stage, fix copying Signed-off-by: Aaron Teo <[email protected]> * devops: remove rustc installation in build Signed-off-by: Aaron Teo <[email protected]> * devops: disable full target for now Signed-off-by: Aaron Teo <[email protected]> * devops: attempting static build Signed-off-by: Aaron Teo <[email protected]> * devops: merge s390x dockerfile into cpu for now Signed-off-by: Aaron Teo <[email protected]> * devops: switch to gcc image for build step Signed-off-by: Aaron Teo <[email protected]> * devops: remove build essentials Signed-off-by: Aaron Teo <[email protected]> * devops: install openblas into base target Signed-off-by: Aaron Teo <[email protected]> * devops: go back to s390x dockerfile Signed-off-by: Aaron Teo <[email protected]> * devops: remove libggml and libblas Signed-off-by: Aaron Teo <[email protected]> * devops: add full target Signed-off-by: Aaron Teo <[email protected]> * devops: add break system packages Signed-off-by: Aaron Teo <[email protected]> * devops: add libjpeg Signed-off-by: Aaron Teo <[email protected]> * devops: add missing cmake dep Signed-off-by: Aaron Teo <[email protected]> * devops: finalise docker images for s390x Signed-off-by: Aaron Teo <[email protected]> * devops: add custom openblas patch Signed-off-by: Aaron Teo <[email protected]> * devops: use libopenblas-dev instead of libopenblas-openmp-dev Signed-off-by: Aaron Teo <[email protected]> * devops: add s390x docker build Signed-off-by: Aaron Teo <[email protected]> --------- Signed-off-by: Aaron Teo <[email protected]>
This Pull Request adds the s390x Llama.cpp Dockerfiles, separate away from the original
cpu.Dockerfileas we have additional dependencies. zDNN backend Dockerfile will be added at a later date.GitHub Actions Workflowdocker.ymlwill not build the s390x containers yet until I can verify that theubuntu-24.04-s390ximage is already made available to the Llama.cpp repository.