Skip to content

Commit 9a0e10c

Browse files
authored
Merge pull request #729 from merveenoyan/small_fix
Small fix to getting started code in README
2 parents 1a5d43e + 9f293ff commit 9a0e10c

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -119,7 +119,7 @@ torch.nn.Embedding(...) -> bnb.nn.StableEmbedding(...) # recommended for NLP mo
119119
```
120120

121121
Note that by default all parameter tensors with less than 4096 elements are kept at 32-bit even if you initialize those parameters with 8-bit optimizers. This is done since such small tensors do not save much memory and often contain highly variable parameters (biases) or parameters that require high precision (batch norm, layer norm). You can change this behavior like so:
122-
```
122+
```python
123123
# parameter tensors with less than 16384 values are optimized in 32-bit
124124
# it is recommended to use multiplies of 4096
125125
adam = bnb.optim.Adam8bit(model.parameters(), min_8bit_size=16384)

0 commit comments

Comments
 (0)