Skip to content

Commit 850a0c7

Browse files
committed
fix
1 parent e51cdf4 commit 850a0c7

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

docs/17_flash_attn/01_flash_attn_v1_part1.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -88,15 +88,15 @@ $$
8888

8989
---
9090

91-
**步骤 1:外层循环 $ j=1 $,处理块 $\mathbf{K}_1, \mathbf{V}_1$**
91+
**步骤 1:外层循环 $ j=1 $,处理块 $\mathbf{K}_1, \mathbf{V}_1$ **
9292

9393
1. **加载 $\mathbf{K}_1, \mathbf{V}_1$ 到 SRAM**
9494

9595
$$
9696
\mathbf{K}_1 = \begin{bmatrix} k_{11} & k_{12} \\ k_{21} & k_{22} \end{bmatrix}, \quad \mathbf{V}_1 = \begin{bmatrix} v_{11} & v_{12} \\ v_{21} & v_{22} \end{bmatrix}
9797
$$
9898

99-
2. **内层循环 $ i=1 $,处理块 $\mathbf{Q}_1$**
99+
2. **内层循环 $ i=1 $,处理块 $\mathbf{Q}_1$ **
100100
- **加载数据**
101101
$$
102102
\mathbf{Q}_1 = \begin{bmatrix} q_{11} & q_{12} \\ q_{21} & q_{22} \end{bmatrix}, \quad \mathbf{O}_1 = \begin{bmatrix} 0 & 0 \\ 0 & 0 \end{bmatrix}, \quad \ell_1 = [0, 0]^T, \quad m_1 = [-\infty, -\infty]^T
@@ -122,7 +122,7 @@ $$
122122
- 类似地,加载 $\mathbf{Q}_2 = \begin{bmatrix} q_{31} & q_{32} \\ q_{41} & q_{42} \end{bmatrix}$,计算 $\mathbf{S}_{21} = \mathbf{Q}_2 \mathbf{K}_1^T$,更新后两行 $\mathbf{O}_2$。
123123

124124

125-
**步骤 2:外层循环 $ j=2 $,处理块 $\mathbf{K}_2, \mathbf{V}_2$**
125+
**步骤 2:外层循环 $ j=2 $,处理块 $\mathbf{K}_2, \mathbf{V}_2$ **
126126

127127
1. **加载 $\mathbf{K}_2, \mathbf{V}_2$ 到 SRAM**
128128
$$
@@ -142,7 +142,7 @@ $$
142142
$$
143143
- **结果等价于全局 Softmax**:最终 $\mathbf{O}_1$ 为前两行注意力结果的加权和。
144144

145-
3. **内层循环 $ i=2 $,处理块 $\mathbf{Q}_2$**
145+
3. **内层循环 $ i=2 $,处理块 $\mathbf{Q}_2$ **
146146
- 类似地,计算 $\mathbf{S}_{22} = \mathbf{Q}_2 \mathbf{K}_2^T$,更新后两行 $\mathbf{O}_2$。
147147

148148

0 commit comments

Comments
 (0)