Skip to content

Conversation

@jtreminio
Copy link

Attempting to use SageAttention3 for an image upscale fails every time due to uniform sequence length requirements, unless I have SA2 as a fallback. However, this was both a silent fallback (this PR adds logging), and I figured we could actually fix the implementation to allow SA3 to work.

  • grouping sequences by length
  • processing each group with SA3
  • reassembling outputs

Changes:

  • uniform path: Unchanged, when all sequences have the same length, uses direct batching
  • variable-length path: groups sequences by length and processes each group with SA3
  • logging: logs when grouped attention is used and when fallback occurs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant