-
Notifications
You must be signed in to change notification settings - Fork 13.3k
Add Trigger at PR creation #15386
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Trigger at PR creation #15386
Conversation
…ctions workflow Removed the legacy .devops/cloud-v-pipeline Jenkins CI configuration and introduced .github/workflows/build-riscv-native.yml for native RISC-V builds using GitHub Actions.
Replace Jenkins Cloud-V pipeline with GitHub Actions RISC-V native build
Since it's building on native it would be useful if it ran tests like the other native builds: llama.cpp/.github/workflows/build.yml Lines 286 to 290 in b143fbc
|
Adding |
Yes. You are right. But since the setup is currently running CIs inside the podman container, which is running on a non-root user on RISC-V, the docker execution is not yet successful. I am testing this and will take some time. I have created this PR since the last PR, with just the building project functionality merged, but without the Also, the RISC-V doesn't seem to have full Vulkan support right now, so all tests as written in build.yml may not run successfully but I am working on this. |
CPU-only is fine, but looking at how long the build takes I'd say |
Just installing |
In case of GitHub-hosted runners, the ccache has to be made to stay consistent, but as this self-hosted runner is the same for every runner, I suppose the ccache is naturally the same for every run. I have added a step to set up a directory of the ccache with a 5 GB cache capacity. |
The podman container is not ephemeral and is always up. So the folder should remain consistent in every run. But I have added a step for ccache in the workflow file. |
Cleanup and fix https://github.com/ggml-org/llama.cpp/actions/runs/17126302903/job/48578719271?pr=15386 and I will merge. |
* Changed the CI file to hw * Changed the CI file to hw * Added to sudoers for apt * Removed the clone command and used checkout * Added libcurl * Added gcc-14 * Checking gcc --version * added gcc-14 symlink * added CC and C++ variables * Added the gguf weight * Changed the weights path * Added system specification * Removed white spaces * ci: Replace Jenkins riscv native build Cloud-V pipeline with GitHub Actions workflow Removed the legacy .devops/cloud-v-pipeline Jenkins CI configuration and introduced .github/workflows/build-riscv-native.yml for native RISC-V builds using GitHub Actions. * removed trailing whitespaces * Added the trigger at PR creation * Corrected OS name * Added ccache as setup package * Added ccache for self-hosted runner * Added directory for ccache size storage Co-authored-by: Sigbjørn Skjæret <[email protected]> * Changed the build command and added ccache debug log * Added the base dir for the ccache * Re-trigger CI * Cleanup and refactored ccache steps * Cleanup and refactored ccache steps --------- Co-authored-by: Akif Ejaz <[email protected]> Co-authored-by: Sigbjørn Skjæret <[email protected]>
sudo apt-get update || true | ||
sudo apt-get install -y --no-install-recommends \ | ||
build-essential \ | ||
gcc-14-riscv64-linux-gnu \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi, Since we are building directly on a RISC-V Linux machine, are we still required to use a cross-compilation toolchain?
It seems that native compilation should be sufficient in this case. Could you help clarify the motivation for using a cross toolchain here? Thnaks you.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the cross-compilation was only introduced because we didnt have the hardware for checking the RVV1.0 support on RISC-V. But I did not add this CI, so author may better tell us about the motivation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Installing gcc-14-riscv64-linux-gnu doesn't install a cross compiler. In fact, this is the native toolchain when the command is executed inside the riscv compute.
Also, the build essential installs 13.2.0 GCC by default, which has a vector intrinsics issue with RISC-V.
Check this issue and comment: #12693 (comment)
@alitariq4589 The CI has been failing for a few days now with this error:
|
Thank you for letting me know. I will resolve this by today. |
@CISC I have checked the latest failing CI and this is not the runner that I integrated with my CI. If you have access to the actions settings, there should be a runner named
I don't know if you can assign a label to the runner from inside the repo settings. If you can, then I can change the CI to use that label. The cause of the issue is that CI workflow has If you cannot change the label, then can you check if there is a unique label which is not common in both |
Also, my version of github runner is 2.321.0. I have patched the GitHub Actions version 2.328.0 for RISC-V from upstream. I am working to update all the running instances to use tha,t but it will take some time. |
Changed the runner label in #16149 . I think the workflow needs approval to run. |
Thanks, I just opened a PR too: #16150. Will merge that one. |
Cool. I will close mine then. |
@CISC, I am adding complete CI for RISC-V, and I would like to know if it is okay if I add the tests to the build.yml file instead of creating a separate file for RISC-V? (I will delete the old CI file once done) |
Yes, please do. |
@CISC There seems to be some issue with the
Check this test. Is this something expected? This seems to be the hugging face repo. Should I create an issue for this? |
Not sure why it's |
Most of the RISC-V chips are little endian and I am assuming k1 (chip on the RISC-V board which I am using for builds) should also be little endian. Is this something directly related to llama.cpp or is this parsed in some other repository? |
I wonder, |
I don't yet understand how this works. I will have to look closely at this CI script to determine which model/file is causing the incorrect magic header. |
It looks like all of them, downloaded from here: https://huggingface.co/ggml-org/vocabs/tree/main |
An update on this bug... There seems to be an issue with how the test runs on the fork. I ran the CI test on the master of my fork (with no added changes, so it stays synced with fork) on my x86 work laptop and got the following error:
I dont see this while running on the upstream repo |
Ahhh, I know, you don't have |
I have checked it. It is installed. |
|
Ok, but it is definitely a So, for some reason LFS files are not resolved. |
Not sure if this is relevant, but git lfs install |
Yes, that was right. It was git-lfs issue. All the tests are now passing except one ggml-ci-riscv64-native-cpu-high-perf. I think it is because of the missing Python transformers module. I am installing it. If this is not installable, then I think I will have to add the condition in the test scripts to convert the weights to the appropriate format and then transfer to RISC-V for execution. |
@ggerganov This is a reference PR to #14439 for adding CI with RVV1.0 hardware
In the previously merged PR, the workflow did not have a
pull_request
flag, so the builds are not getting triggered. I have just added thepull_request
flag.I am working on creating a file that mimics the functionality of the build.yml file, which contains all the tests that can run on RISC-V. Until then, please consider this change so that the project can just build on RISC-V vector hardware for now.