Skip to content

Conversation

@taronaeo
Copy link
Collaborator

@taronaeo taronaeo commented Sep 10, 2025

This Pull Request adds the s390x Llama.cpp Dockerfiles, separate away from the original cpu.Dockerfile as we have additional dependencies. zDNN backend Dockerfile will be added at a later date.

GitHub Actions Workflow docker.yml will not build the s390x containers yet until I can verify that the ubuntu-24.04-s390x image is already made available to the Llama.cpp repository.

Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
failure to load model

Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
This reverts commit 0a7664a.

Signed-off-by: Aaron Teo <[email protected]>
Signed-off-by: Aaron Teo <[email protected]>
(cherry picked from commit 0a7664a)
Signed-off-by: Aaron Teo <[email protected]>
@taronaeo taronaeo requested a review from ngxson as a code owner September 10, 2025 11:09
@github-actions github-actions bot added the devops improvements to build systems and github actions label Sep 10, 2025
@taronaeo taronaeo requested review from CISC and slaren September 11, 2025 15:24
@taronaeo
Copy link
Collaborator Author

taronaeo commented Sep 20, 2025

I notice that my patched OpenBLAS has slower performance for Prompt Processing but faster performance for Token Generation. This doesn't look right, I might revert back to the distributed OpenBLAS instead.

@taronaeo
Copy link
Collaborator Author

taronaeo commented Sep 22, 2025

Reverted back to using libopenblas-dev as the performance difference isn't that much compared to my custom patch, and lesser maintenance cost.

model size params backend threads test t/s (libopenblas-openmp-dev) t/s (libopenblas-dev) t/s (custom OpenBLAS fork w/ cache patch)
granite 3B Q4_K - Medium 1.44 GiB 2.53 B BLAS 8 pp512 29.02 ± 0.43 60.95 ± 0.05 55.95 ± 0.13
granite 3B Q4_K - Medium 1.44 GiB 2.53 B BLAS 8 tg128 20.72 ± 0.01 21.89 ± 0.04 22.50 ± 0.20

Also, I've added s390x docker build to docker.yml. If it takes an unreasonable amount of time to build, I will switch it to use the actual ubuntu-22.04-s390x or ubuntu-24.04-s390x image. Re-requesting review in-case I missed anything.

Edit: My custom OpenBLAS patch is way faster without Docker. But I can't seem to replicate the same performance oh well.

model size params backend threads test t/s
granite 3B Q4_K - Medium 1.44 GiB 2.53 B BLAS 8 pp512 81.98 ± 0.04
granite 3B Q4_K - Medium 1.44 GiB 2.53 B BLAS 8 tg128 21.61 ± 0.02

@taronaeo taronaeo requested a review from slaren September 22, 2025 06:30
@taronaeo taronaeo merged commit 4b9f4cb into ggml-org:master Sep 23, 2025
3 of 4 checks passed
gabe-l-hart added a commit to gabe-l-hart/llama.cpp that referenced this pull request Sep 23, 2025
* origin/master: (39 commits)
ci : disable AMD workflows + update NVIDIA workflows (ggml-org#16200)
ci : enable Vulkan workflow on Mac (ggml-org#16194)
ggml-cpu: Respect cpumask settings (ggml-org#16164)
ggml : fix uninitialized is_on_grid in quantize_row_iq3_xxs_impl (ggml-org#15928)
zdnn: refactor codebase + add docs (ggml-org#16178)
codeowners : add @danbev to model-conversion example [no ci] (ggml-org#16190)
devops: add s390x containers (ggml-org#15915)
ggml-cpu : fix typo in gemm comments [no ci] (ggml-org#16189)
feat: Add conversion support in GraniteHybrid for non-hybrid (all attn) (ggml-org#16177)
clang-tidy : disable warning about performance enum size (ggml-org#16127)
ggml : implement set_rows with i32 index (ggml-org#16159)
codeowners : update + cleanup (ggml-org#16174)
common : enable `--offline` mode without curl support (ggml-org#16137)
webui : fix handling incomplete chunks (ggml-org#16107)
embedding : fix typos in README (ggml-org#16171)
common : remove unused local variables (ggml-org#16140)
ggml : extend ggml_can_fuse to work with non-sequential nodes (ggml-org#16123)
ggml : add ggml_op_is_empty (ggml-org#16122)
codeowners : update ownership for @ngxson and @allozuar (ggml-org#16128)
Vulkan: add conv_transpose_2d operation (ggml-org#16022)
...
struct pushed a commit to struct/llama.cpp that referenced this pull request Sep 26, 2025
* devops: add s390x dockerfile

Signed-off-by: Aaron Teo <[email protected]>

* devops: add missing ninja

Signed-off-by: Aaron Teo <[email protected]>

* devops: move s390x docker into cpu docker

Signed-off-by: Aaron Teo <[email protected]>

* devops: rework s390x docker

Signed-off-by: Aaron Teo <[email protected]>

* devops: copy more tools

Signed-off-by: Aaron Teo <[email protected]>

* devops: add server build step

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove apt clean steps as distroless misses it

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove apt commands from distroless

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix shared libs in distroless

Signed-off-by: Aaron Teo <[email protected]>

* devops: use correct libs path

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix shared libs

Signed-off-by: Aaron Teo <[email protected]>

* devops: add collector stage

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix missing stage ref

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix permission issue

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix unknown model loading failures

Signed-off-by: Aaron Teo <[email protected]>

* devops: attempt at fixing model loading failure

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix missing ggml shared object

failure to load model

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove move shared objects

Signed-off-by: Aaron Teo <[email protected]>

* devops: move libggml-cpu and blas into bin

Signed-off-by: Aaron Teo <[email protected]>

* devops: finalise hardened server stage

Signed-off-by: Aaron Teo <[email protected]>

* devops: add cli target

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix typos

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix missing shared libraries in base

Signed-off-by: Aaron Teo <[email protected]>

* devops: update debian target

Signed-off-by: Aaron Teo <[email protected]>

* devops: formalise llama.cpp loc

Signed-off-by: Aaron Teo <[email protected]>

* Revert "devops: formalise llama.cpp loc"

This reverts commit 0a7664a.

Signed-off-by: Aaron Teo <[email protected]>

* devops: formalise llama.cpp loc

Signed-off-by: Aaron Teo <[email protected]>
(cherry picked from commit 0a7664a)
Signed-off-by: Aaron Teo <[email protected]>

* devops: attempt at fixing missing dir

Signed-off-by: Aaron Teo <[email protected]>

* devops: attempt at making it cache the build

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix copying process

Signed-off-by: Aaron Teo <[email protected]>

* devops: make build dir an argument

Signed-off-by: Aaron Teo <[email protected]>

* Revert "devops: make build dir an argument"

This reverts commit 4386989.

Signed-off-by: Aaron Teo <[email protected]>

* devops: add build stage for gguf-py

Signed-off-by: Aaron Teo <[email protected]>

* devops: move gguf-py installation into build stage

Signed-off-by: Aaron Teo <[email protected]>

* devops: break system packages?

Signed-off-by: Aaron Teo <[email protected]>

* devops: add rust compiler installer

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix rustc not found

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove cache mount to allow rustc to persist

Signed-off-by: Aaron Teo <[email protected]>

* devops: move rustc installation to another layer

Signed-off-by: Aaron Teo <[email protected]>

* devops: move gguf-py installation to full stage, fix copying

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove rustc installation in build

Signed-off-by: Aaron Teo <[email protected]>

* devops: disable full target for now

Signed-off-by: Aaron Teo <[email protected]>

* devops: attempting static build

Signed-off-by: Aaron Teo <[email protected]>

* devops: merge s390x dockerfile into cpu for now

Signed-off-by: Aaron Teo <[email protected]>

* devops: switch to gcc image for build step

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove build essentials

Signed-off-by: Aaron Teo <[email protected]>

* devops: install openblas into base target

Signed-off-by: Aaron Teo <[email protected]>

* devops: go back to s390x dockerfile

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove libggml and libblas

Signed-off-by: Aaron Teo <[email protected]>

* devops: add full target

Signed-off-by: Aaron Teo <[email protected]>

* devops: add break system packages

Signed-off-by: Aaron Teo <[email protected]>

* devops: add libjpeg

Signed-off-by: Aaron Teo <[email protected]>

* devops: add missing cmake dep

Signed-off-by: Aaron Teo <[email protected]>

* devops: finalise docker images for s390x

Signed-off-by: Aaron Teo <[email protected]>

* devops: add custom openblas patch

Signed-off-by: Aaron Teo <[email protected]>

* devops: use libopenblas-dev instead of libopenblas-openmp-dev

Signed-off-by: Aaron Teo <[email protected]>

* devops: add s390x docker build

Signed-off-by: Aaron Teo <[email protected]>

---------

Signed-off-by: Aaron Teo <[email protected]>
yael-works pushed a commit to yael-works/llama.cpp that referenced this pull request Oct 15, 2025
* devops: add s390x dockerfile

Signed-off-by: Aaron Teo <[email protected]>

* devops: add missing ninja

Signed-off-by: Aaron Teo <[email protected]>

* devops: move s390x docker into cpu docker

Signed-off-by: Aaron Teo <[email protected]>

* devops: rework s390x docker

Signed-off-by: Aaron Teo <[email protected]>

* devops: copy more tools

Signed-off-by: Aaron Teo <[email protected]>

* devops: add server build step

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove apt clean steps as distroless misses it

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove apt commands from distroless

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix shared libs in distroless

Signed-off-by: Aaron Teo <[email protected]>

* devops: use correct libs path

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix shared libs

Signed-off-by: Aaron Teo <[email protected]>

* devops: add collector stage

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix missing stage ref

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix permission issue

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix unknown model loading failures

Signed-off-by: Aaron Teo <[email protected]>

* devops: attempt at fixing model loading failure

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix missing ggml shared object

failure to load model

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove move shared objects

Signed-off-by: Aaron Teo <[email protected]>

* devops: move libggml-cpu and blas into bin

Signed-off-by: Aaron Teo <[email protected]>

* devops: finalise hardened server stage

Signed-off-by: Aaron Teo <[email protected]>

* devops: add cli target

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix typos

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix missing shared libraries in base

Signed-off-by: Aaron Teo <[email protected]>

* devops: update debian target

Signed-off-by: Aaron Teo <[email protected]>

* devops: formalise llama.cpp loc

Signed-off-by: Aaron Teo <[email protected]>

* Revert "devops: formalise llama.cpp loc"

This reverts commit 0a7664a.

Signed-off-by: Aaron Teo <[email protected]>

* devops: formalise llama.cpp loc

Signed-off-by: Aaron Teo <[email protected]>
(cherry picked from commit 0a7664a)
Signed-off-by: Aaron Teo <[email protected]>

* devops: attempt at fixing missing dir

Signed-off-by: Aaron Teo <[email protected]>

* devops: attempt at making it cache the build

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix copying process

Signed-off-by: Aaron Teo <[email protected]>

* devops: make build dir an argument

Signed-off-by: Aaron Teo <[email protected]>

* Revert "devops: make build dir an argument"

This reverts commit 4386989.

Signed-off-by: Aaron Teo <[email protected]>

* devops: add build stage for gguf-py

Signed-off-by: Aaron Teo <[email protected]>

* devops: move gguf-py installation into build stage

Signed-off-by: Aaron Teo <[email protected]>

* devops: break system packages?

Signed-off-by: Aaron Teo <[email protected]>

* devops: add rust compiler installer

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix rustc not found

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove cache mount to allow rustc to persist

Signed-off-by: Aaron Teo <[email protected]>

* devops: move rustc installation to another layer

Signed-off-by: Aaron Teo <[email protected]>

* devops: move gguf-py installation to full stage, fix copying

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove rustc installation in build

Signed-off-by: Aaron Teo <[email protected]>

* devops: disable full target for now

Signed-off-by: Aaron Teo <[email protected]>

* devops: attempting static build

Signed-off-by: Aaron Teo <[email protected]>

* devops: merge s390x dockerfile into cpu for now

Signed-off-by: Aaron Teo <[email protected]>

* devops: switch to gcc image for build step

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove build essentials

Signed-off-by: Aaron Teo <[email protected]>

* devops: install openblas into base target

Signed-off-by: Aaron Teo <[email protected]>

* devops: go back to s390x dockerfile

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove libggml and libblas

Signed-off-by: Aaron Teo <[email protected]>

* devops: add full target

Signed-off-by: Aaron Teo <[email protected]>

* devops: add break system packages

Signed-off-by: Aaron Teo <[email protected]>

* devops: add libjpeg

Signed-off-by: Aaron Teo <[email protected]>

* devops: add missing cmake dep

Signed-off-by: Aaron Teo <[email protected]>

* devops: finalise docker images for s390x

Signed-off-by: Aaron Teo <[email protected]>

* devops: add custom openblas patch

Signed-off-by: Aaron Teo <[email protected]>

* devops: use libopenblas-dev instead of libopenblas-openmp-dev

Signed-off-by: Aaron Teo <[email protected]>

* devops: add s390x docker build

Signed-off-by: Aaron Teo <[email protected]>

---------

Signed-off-by: Aaron Teo <[email protected]>
pwilkin pushed a commit to pwilkin/llama.cpp that referenced this pull request Oct 23, 2025
* devops: add s390x dockerfile

Signed-off-by: Aaron Teo <[email protected]>

* devops: add missing ninja

Signed-off-by: Aaron Teo <[email protected]>

* devops: move s390x docker into cpu docker

Signed-off-by: Aaron Teo <[email protected]>

* devops: rework s390x docker

Signed-off-by: Aaron Teo <[email protected]>

* devops: copy more tools

Signed-off-by: Aaron Teo <[email protected]>

* devops: add server build step

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove apt clean steps as distroless misses it

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove apt commands from distroless

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix shared libs in distroless

Signed-off-by: Aaron Teo <[email protected]>

* devops: use correct libs path

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix shared libs

Signed-off-by: Aaron Teo <[email protected]>

* devops: add collector stage

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix missing stage ref

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix permission issue

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix unknown model loading failures

Signed-off-by: Aaron Teo <[email protected]>

* devops: attempt at fixing model loading failure

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix missing ggml shared object

failure to load model

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove move shared objects

Signed-off-by: Aaron Teo <[email protected]>

* devops: move libggml-cpu and blas into bin

Signed-off-by: Aaron Teo <[email protected]>

* devops: finalise hardened server stage

Signed-off-by: Aaron Teo <[email protected]>

* devops: add cli target

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix typos

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix missing shared libraries in base

Signed-off-by: Aaron Teo <[email protected]>

* devops: update debian target

Signed-off-by: Aaron Teo <[email protected]>

* devops: formalise llama.cpp loc

Signed-off-by: Aaron Teo <[email protected]>

* Revert "devops: formalise llama.cpp loc"

This reverts commit 0a7664a.

Signed-off-by: Aaron Teo <[email protected]>

* devops: formalise llama.cpp loc

Signed-off-by: Aaron Teo <[email protected]>
(cherry picked from commit 0a7664a)
Signed-off-by: Aaron Teo <[email protected]>

* devops: attempt at fixing missing dir

Signed-off-by: Aaron Teo <[email protected]>

* devops: attempt at making it cache the build

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix copying process

Signed-off-by: Aaron Teo <[email protected]>

* devops: make build dir an argument

Signed-off-by: Aaron Teo <[email protected]>

* Revert "devops: make build dir an argument"

This reverts commit 4386989.

Signed-off-by: Aaron Teo <[email protected]>

* devops: add build stage for gguf-py

Signed-off-by: Aaron Teo <[email protected]>

* devops: move gguf-py installation into build stage

Signed-off-by: Aaron Teo <[email protected]>

* devops: break system packages?

Signed-off-by: Aaron Teo <[email protected]>

* devops: add rust compiler installer

Signed-off-by: Aaron Teo <[email protected]>

* devops: fix rustc not found

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove cache mount to allow rustc to persist

Signed-off-by: Aaron Teo <[email protected]>

* devops: move rustc installation to another layer

Signed-off-by: Aaron Teo <[email protected]>

* devops: move gguf-py installation to full stage, fix copying

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove rustc installation in build

Signed-off-by: Aaron Teo <[email protected]>

* devops: disable full target for now

Signed-off-by: Aaron Teo <[email protected]>

* devops: attempting static build

Signed-off-by: Aaron Teo <[email protected]>

* devops: merge s390x dockerfile into cpu for now

Signed-off-by: Aaron Teo <[email protected]>

* devops: switch to gcc image for build step

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove build essentials

Signed-off-by: Aaron Teo <[email protected]>

* devops: install openblas into base target

Signed-off-by: Aaron Teo <[email protected]>

* devops: go back to s390x dockerfile

Signed-off-by: Aaron Teo <[email protected]>

* devops: remove libggml and libblas

Signed-off-by: Aaron Teo <[email protected]>

* devops: add full target

Signed-off-by: Aaron Teo <[email protected]>

* devops: add break system packages

Signed-off-by: Aaron Teo <[email protected]>

* devops: add libjpeg

Signed-off-by: Aaron Teo <[email protected]>

* devops: add missing cmake dep

Signed-off-by: Aaron Teo <[email protected]>

* devops: finalise docker images for s390x

Signed-off-by: Aaron Teo <[email protected]>

* devops: add custom openblas patch

Signed-off-by: Aaron Teo <[email protected]>

* devops: use libopenblas-dev instead of libopenblas-openmp-dev

Signed-off-by: Aaron Teo <[email protected]>

* devops: add s390x docker build

Signed-off-by: Aaron Teo <[email protected]>

---------

Signed-off-by: Aaron Teo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devops improvements to build systems and github actions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants