Skip to content

Shape of bias? #8

@Jostarndt

Description

@Jostarndt

Hi! What a cool project!

I wanted to know if you could add the shape of the bias to the readme? The readme uses "None"; but maybe there is support also for the bias already?

And is the bias the bias which has been used in ViT? So softmax(Q*K + B)* V ? So Bias should be of shape[batch, window_size ** 2, window_size ** 2]?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions