Commit b22a209
committed
Fix chain rule in HopeNetwork backward pass
The backward pass was incorrectly breaking the chain rule by:
- Iterating over learning levels instead of actual CMS blocks
- Using modulo indexing (level % _numCMSLevels) which broke gradient flow
- Reusing the same gradient for all blocks instead of chaining them
- Accumulating gradients incorrectly
Fixed by:
- Processing context flow gradients in reverse, accumulating them into upstream gradient
- Iterating CMS blocks in reverse order (last to first) without modulo
- Properly chaining gradients: each block receives accumulated gradient from previous block
- Returning final chained gradient as true derivative w.r.t. HOPE input
This ensures proper backpropagation through the entire HOPE architecture.1 parent 94398a6 commit b22a209
1 file changed
+15
-15
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
174 | 174 | | |
175 | 175 | | |
176 | 176 | | |
177 | | - | |
178 | | - | |
179 | | - | |
| 177 | + | |
| 178 | + | |
180 | 179 | | |
181 | 180 | | |
182 | | - | |
| 181 | + | |
183 | 182 | | |
| 183 | + | |
184 | 184 | | |
185 | | - | |
186 | | - | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
187 | 188 | | |
188 | | - | |
189 | | - | |
190 | | - | |
191 | | - | |
192 | | - | |
193 | | - | |
194 | | - | |
195 | | - | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
196 | 196 | | |
197 | 197 | | |
198 | | - | |
| 198 | + | |
199 | 199 | | |
200 | 200 | | |
201 | 201 | | |
| |||
0 commit comments