Skip to content

Conversation

MollySophia
Copy link
Collaborator

This seems to be a mistake that wasn't discovered in the initial PR.
The time_decay operations in RWKV v6 is pretty sensitive (since it's done recursively many times during the sequence). It's better keeps these related weights unquantized.

@github-actions github-actions bot added the python python script changes label Sep 9, 2024
@BlinkDL
Copy link

BlinkDL commented Sep 9, 2024

@MollySophia can add time_maa_w1 time_maa_w2 to exclusion list too?

and f16 is enough

@MollySophia
Copy link
Collaborator Author

@MollySophia can add time_maa_w1 time_maa_w2 to exclusion list too?

and f16 is enough

These two are already there :D

@ggerganov ggerganov merged commit 0b4ac75 into ggml-org:master Sep 10, 2024
54 checks passed
dsx1986 pushed a commit to dsx1986/llama.cpp that referenced this pull request Oct 29, 2024
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Nov 15, 2024
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python python script changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants