Skip to content

Commit c378245

Browse files
authored
add blackwell attentions (#383)
1 parent 206b93c commit c378245

File tree

3 files changed

+21
-0
lines changed

3 files changed

+21
-0
lines changed
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
bf16_flash_attention_fwd_shape1:
2+
op:
3+
blackwell_attentions
4+
args:
5+
--op blackwell_attentions --seq-len 8192 --batch 8 --n-heads 16 --d-head 128 --rep 3000 --sleep 1.0 --metrics tflops --simple-output --only gluon_blackwell_tutorial_persistent_fwd
6+
envs:
7+
WITH_GLUON=1
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
bf16_flash_attention_fwd_shape2:
2+
op:
3+
blackwell_attentions
4+
args:
5+
--op blackwell_attentions --seq-len 1024 --batch 1152 --n-heads 4 --d-head 128 --rep 3000 --sleep 1.0 --metrics tflops --simple-output --only gluon_blackwell_tutorial_persistent_fwd
6+
envs:
7+
WITH_GLUON=1
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
bf16_flash_attention_fwd_shape2:
2+
op:
3+
blackwell_attentions
4+
args:
5+
--op blackwell_attentions --seq-len 8192 --batch 4 --n-heads 32 --d-head 128 --rep 3000 --sleep 1.0 --metrics tflops --simple-output --only gluon_blackwell_tutorial_persistent_fwd
6+
envs:
7+
WITH_GLUON=1

0 commit comments

Comments
 (0)