Skip to content

Commit 2e69c3d

Browse files
committed
Adds dbias accumulation flag to bwd params
Introduces a toggle to optionally accumulate bias gradients during backward attention. Enables skipping unnecessary dbias work when unused and provides clearer control for kernels, aiding performance and configurability.
1 parent 623b75d commit 2e69c3d

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

csrc/flash_dmattn/src/flash.h

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -195,6 +195,8 @@ struct Flash_bwd_params : public Flash_fwd_params {
195195

196196
bool deterministic;
197197
index_t dq_accum_split_stride;
198+
199+
bool accum_dbias;
198200
};
199201

200202
////////////////////////////////////////////////////////////////////////////////////////////////////

0 commit comments

Comments
 (0)