Skip to content

Commit 7f7cbe2

Browse files
authored
Update README.md
1 parent d9212c2 commit 7f7cbe2

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ This improvement in training speed has been brought about by the following techn
2626
* Cautious Weight Decay w/ schedule tied to LR
2727
* Exponential decay of residual stream
2828
* Batch size schedule
29+
* Max seq length schedule
2930
* Partial Key Offset
3031
* Multi token prediction
3132
* Untie embed and lm_head at 2/3 of training

0 commit comments

Comments
 (0)