Discussions

• 1 You must be logged in to vote

  🙌

  Routed Attention: Learning when to use O(N) conv vs O(N²) attention (75-99% compute savings)

  MikeyBeez started Feb 7, 2026 in Show and tell

  

  

  0
• 4 You must be logged in to vote

  💬

  Hope to update to CUDA13

  zihugithub started Oct 29, 2025 in General

  

  

  1
• 1 You must be logged in to vote

  🙏

  Why does FA3 use persistent blocks while FA2 doesn't?

  MoooonkIn asked Dec 31, 2025 in Q&A · Answered

  

  

  3
• 2 You must be logged in to vote

  🙏

  When can I use the logsumexp I get from return_attn_probs?

  ad8e asked Dec 30, 2025 in Q&A · Answered

  

  

  1
• 1 You must be logged in to vote

  💬

  when will rtx50 series GPU be supported?

  nianqiu-liu started Mar 27, 2025 in General

  

  

  

  

  6
• 3 You must be logged in to vote

  💬

  Hopper FP8 Blockwise Attention

  ipiszy started Jun 2, 2025 in General

  

  

  

  

  2
• 1 You must be logged in to vote

  🙏

  Clarification: baseline GLIBC version for 2.8.x releases

  luisbarrancos asked Nov 14, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💡

  Extending FlashAttention to Hyperbolic (Lorentzian) Space — Flash Lorentz Attention

  Dominator-Index started Oct 16, 2025 in Ideas

  

  

  0
• 1 You must be logged in to vote

  🙏

  Is there a flash-attention for win_amd64?

  yapuppycat asked Aug 19, 2025 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  💬

  "Why Low-Precision Transformer Training Fails: An Analysis on Flash Attention"

  AustenMan started Oct 10, 2025 in General

  

  

  2
• 1 You must be logged in to vote

  🙏

  3090 install 3.0.0b1

  Dimonye0531 asked Aug 12, 2025 in Q&A · Closed · Unanswered

  

  

  

  

  4
• 1 You must be logged in to vote

  🙏

  Why does the volume of HBM traffic not change when observed with NCU?

  rajesh-s asked Sep 11, 2025 in Q&A · Closed · Answered

  

  

  4
• 1 You must be logged in to vote

  🙏

  Clarification on fused kernel behavior for FP8 vs. BF16

  EricJKrebs asked Sep 4, 2025 in Q&A · Unanswered

  

  

  2
• 2 You must be logged in to vote

  💬

  Interactive code diagram

  briandiloreto started Aug 28, 2025 in General

  

  

  0
• 3 You must be logged in to vote

  🙌

  Prebuilt wheels provided via 3rd party repository

  MiroPsota started Aug 24, 2025 in Show and tell

  

  

  0
• 1 You must be logged in to vote

  🙏

  AMD FA performance

  manu-web asked Jun 12, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  What is the latest version that supports the Turing architecture (if any)

  CoMelissant asked May 27, 2025 in Q&A · Unanswered

  

  

  0
• 3 You must be logged in to vote

  💡

  int4 forward for flash attention?

  DhruvaBansal00 started Mar 14, 2025 in Ideas

  

  

  3
• 1 You must be logged in to vote

  🙏

  Install 2.7.3 and 3.0.0b1 in same virtual environment

  quanta42 asked Jan 21, 2025 in Q&A · Unanswered

  

  

  2
• 1 You must be logged in to vote

  💡

  How to Extend FlashAttention to Nearly Infinite HeadDim and Achieve Fully Fused MLA?

  DefTruth started Feb 5, 2025 in Ideas

  

  

  1
• 1 You must be logged in to vote

  🙌

  For anyone having issues with CMD length, PIP, or flash-attn building in general, try PowerShellPython

  leomaxwell973 started Feb 8, 2025 in Show and tell

  

  

  0