Skip to content

Conversation

@zhiyuan1i
Copy link
Contributor

Make sure to read the contributing guidelines before submitting a PR

This PR aims to enhance support for RWKV6 models, it enables wkv6 at Vulkan backend.

@github-actions github-actions bot added Vulkan Issues specific to the Vulkan backend ggml changes relating to the ggml tensor library for machine learning labels Dec 14, 2024
@0cc4m 0cc4m self-requested a review December 15, 2024 11:27
@zhiyuan1i
Copy link
Contributor Author

image
image
update with tests :)

Copy link
Collaborator

@jeffbolznv jeffbolznv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with rwkv, but the shader code appears to correctly match CUDA.

@jeffbolznv
Copy link
Collaborator

Thanks for the contribution. I'll try to test this locally tomorrow.

Copy link
Collaborator

@jeffbolznv jeffbolznv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the changes. LGTM. I ran the backend tests on RTX 4070 and they passed.

Copy link
Collaborator

@0cc4m 0cc4m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, great work!

@0cc4m 0cc4m merged commit 160bc03 into ggml-org:master Dec 16, 2024
48 checks passed
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Dec 20, 2024
* rwkv_wkv6 vulkan shader

* RWKV_WKV6 Vulkan op tests passed

Signed-off-by: Molly Sophia <[email protected]>

* Apply code format changes

Signed-off-by: Molly Sophia <[email protected]>

* add [[unroll]] and remove unnecessary conditions

* add uma support

* fix erros in EditorConfig Checker

---------

Signed-off-by: Molly Sophia <[email protected]>
Co-authored-by: Molly Sophia <[email protected]>
tinglou pushed a commit to tinglou/llama.cpp that referenced this pull request Feb 13, 2025
* rwkv_wkv6 vulkan shader

* RWKV_WKV6 Vulkan op tests passed

Signed-off-by: Molly Sophia <[email protected]>

* Apply code format changes

Signed-off-by: Molly Sophia <[email protected]>

* add [[unroll]] and remove unnecessary conditions

* add uma support

* fix erros in EditorConfig Checker

---------

Signed-off-by: Molly Sophia <[email protected]>
Co-authored-by: Molly Sophia <[email protected]>
mglambda pushed a commit to mglambda/llama.cpp that referenced this pull request Mar 8, 2025
* rwkv_wkv6 vulkan shader

* RWKV_WKV6 Vulkan op tests passed

Signed-off-by: Molly Sophia <[email protected]>

* Apply code format changes

Signed-off-by: Molly Sophia <[email protected]>

* add [[unroll]] and remove unnecessary conditions

* add uma support

* fix erros in EditorConfig Checker

---------

Signed-off-by: Molly Sophia <[email protected]>
Co-authored-by: Molly Sophia <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning Vulkan Issues specific to the Vulkan backend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants