Skip to content

Commit a646006

Browse files
authored
vulkan: disable FA mask_opt on GCN to improve performance (ggml-org#24362)
* vulkan: disable FA mask_opt on GCN to improve performance * reenable mask opt over attention head size 256
1 parent 167d057 commit a646006

1 file changed

Lines changed: 2 additions & 1 deletion

File tree

ggml/src/ggml-vulkan/ggml-vulkan.cpp

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10310,7 +10310,8 @@ static void ggml_vk_flash_attn(ggml_backend_vk_context * ctx, vk_context& subctx
1031010310
}
1031110311

1031210312
// Only use mask opt when the mask is fairly large. This hasn't been tuned extensively.
10313-
bool use_mask_opt = mask && nem1 >= 32 && nem0 * nem1 > 32768 && nem0 >= tuning_params.block_cols * 16;
10313+
bool use_mask_opt = mask && nem1 >= 32 && nem0 * nem1 > 32768 && nem0 >= tuning_params.block_cols * 16
10314+
&& (ctx->device->architecture != vk_device_architecture::AMD_GCN || HSK > 256 || HSV > 256);
1031410315
vk_fa_pipeline_state fa_pipeline_state = get_fa_pipeline_state(ctx->device, tuning_params, HSK, HSV, aligned, f32acc,
1031510316
mask != nullptr, use_mask_opt, logit_softcap != 0, k->type, v->type);
1031610317

0 commit comments

Comments
 (0)