Skip to content

Commit 5a6afcc

Browse files
authored
resolve diff for mixtral8x7B configs (ROCm#437)
Signed-off-by: Divakar Verma <[email protected]>
1 parent 39456f3 commit 5a6afcc

File tree

3 files changed

+5
-5
lines changed

3 files changed

+5
-5
lines changed

vllm/model_executor/layers/fused_moe/configs/E=8,N=14336,device_name=AMD_Instinct_MI300X.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@
9696
"num_stages": 2,
9797
"waves_per_eu": 0,
9898
"matrix_instr_nonkdim": 16,
99-
"kpack": 1
99+
"kpack": 2
100100
},
101101
"96": {
102102
"BLOCK_SIZE_M": 32,
@@ -123,7 +123,7 @@
123123
"256": {
124124
"BLOCK_SIZE_M": 128,
125125
"BLOCK_SIZE_N": 128,
126-
"BLOCK_SIZE_K": 128,
126+
"BLOCK_SIZE_K": 64,
127127
"GROUP_SIZE_M": 4,
128128
"num_warps": 8,
129129
"num_stages": 2,

vllm/model_executor/layers/fused_moe/configs/E=8,N=1792,device_name=AMD_Instinct_MI300X.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
"num_stages": 2,
99
"waves_per_eu": 0,
1010
"matrix_instr_nonkdim": 16,
11-
"kpack": 1
11+
"kpack": 2
1212
},
1313
"2": {
1414
"BLOCK_SIZE_M": 16,

vllm/model_executor/layers/fused_moe/configs/E=8,N=7168,device_name=AMD_Instinct_MI300X.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
"num_stages": 2,
99
"waves_per_eu": 0,
1010
"matrix_instr_nonkdim": 16,
11-
"kpack": 1
11+
"kpack": 2
1212
},
1313
"2": {
1414
"BLOCK_SIZE_M": 16,
@@ -63,7 +63,7 @@
6363
"num_stages": 2,
6464
"waves_per_eu": 0,
6565
"matrix_instr_nonkdim": 16,
66-
"kpack": 2
66+
"kpack": 1
6767
},
6868
"32": {
6969
"BLOCK_SIZE_M": 16,

0 commit comments

Comments
 (0)