Skip to content

Commit 41412a1

Browse files
authored
Merge pull request #25 from ng-jk/patch-2
Update 01_long_context_processing.md
2 parents a416a67 + bd6b481 commit 41412a1

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

00_COURSE/02_context_processing/01_long_context_processing.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1335,6 +1335,8 @@ class MultiHeadHierarchicalAttention(nn.Module):
13351335

13361336
return global_output
13371337

1338+
```
1339+
13381340
**Ground-up Explanation**: This hierarchical memory system works like a sophisticated filing system in your brain. Working memory is your desk - limited space but immediate access. Short-term memory is like your desk drawers - more space but requires compression. Long-term memory is like your filing cabinets - vast storage but highly organized and compressed. Episodic memory is like your journal of important events.
13391341

13401342
The attention mechanism is like having different types of reading strategies. For short texts, you read every word carefully (standard attention). For very long documents, you read some sections in detail (local windows) while skimming for key points throughout (sparse global attention).

0 commit comments

Comments
 (0)