Skip to content

Evaluate using PyTorch's FlashAttention #363

@swahtz

Description

@swahtz

We should evaluate using Pytorch's built-in Flash Attention operator which also is compatible with their Nested Tensor which we could trivially wrap our JaggedTensor with. We should use Flash Attention when hardware and data types allow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    core libraryCore fVDB library. i.e. anything in the _Cpp module (C++) or fvdb python module

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions