Skip to content

ggml webgpu: support for rope,div,sub,glu,scale,cont operators #16187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 15 commits into from
Sep 30, 2025
Merged

ggml webgpu: support for rope,div,sub,glu,scale,cont operators #16187

merged 15 commits into from
Sep 30, 2025

Conversation

reeselevine
Copy link
Collaborator

Adds support for: ROPE, DIV, SUB, GLU, SCALE, CONT

  • mostly adds new shader code
  • refactored some of the shader templates so that binary operations (add/mul/div/sub) share a common base
  • uses cpy shader for cont
  • I also modified some of the tests in test-backend-ops to add support for inplace testing, which is useful since WebGPU requires that buffer bindings do not alias. This adds minimal runtime to test-backend-ops, since I add about one inplace test per operator variant.

@reeselevine reeselevine requested a review from slaren as a code owner September 23, 2025 01:20
@github-actions github-actions bot added testing Everything test related python python script changes ggml changes relating to the ggml tensor library for machine learning labels Sep 23, 2025
slaren
slaren reviewed Sep 24, 2025
View reviewed changes
ggerganov
ggerganov reviewed Sep 29, 2025
View reviewed changes
@reeselevine
Copy link
Collaborator Author

The failure in the macOS-latest-cmake-arm64{-webgpu} GIthub actions macOS CI are due to timeouts in test-barrier, which I have been seeing occasionally, I think mostly since #15938 and #16029 were merged. It seems to affect both WebGPU and Metal runs.

ggerganov
ggerganov approved these changes Sep 30, 2025
View reviewed changes
@reeselevine reeselevine merged commit 8d78cd2 into ggml-org:master Sep 30, 2025
66 of 70 checks passed
yael-works pushed a commit to yael-works/llama.cpp that referenced this pull request Oct 15, 2025
@reeselevine @ggerganov @yael-works
ggml webgpu: support for rope,div,sub,glu,scale,cont operators (ggml-…
b972b61 
…org#16187)

* Work on rope

* Simplify inplace operation generation and combine mul/add generation

* Work on rope variants

* implement neox rope

* rope complete

* Add sub,div,glu operators

* implement scale op

* Update cpy shader to handle cont/more types

* formatting

* Update test vars printing for rope,rms_norm

* Avoid ROPE hardcoded constants

* Add TODO to change ROPE constants to enum

Co-authored-by: Georgi Gerganov <[email protected]>

* fix TODO comment

---------

Co-authored-by: Georgi Gerganov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@slaren slaren slaren left review comments

@ggerganov ggerganov ggerganov approved these changes

Assignees

No one assigned

Labels

ggml changes relating to the ggml tensor library for machine learning python python script changes testing Everything test related

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@reeselevine @ggerganov @slaren