-
Notifications
You must be signed in to change notification settings - Fork 4
Open
Description
Hi! What a cool project!
I wanted to know if you could add the shape of the bias to the readme? The readme uses "None"; but maybe there is support also for the bias already?
And is the bias the bias which has been used in ViT? So softmax(Q*K + B)* V ? So Bias should be of shape[batch, window_size ** 2, window_size ** 2]?
Metadata
Metadata
Assignees
Labels
No labels