Skip to content

Commit 9f293ff

Browse files
authored
Small fix to README
1 parent 18e827d commit 9f293ff

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ python setup.py install
3838
```python
3939
from transformers import AutoModelForCausalLM
4040
model = AutoModelForCausalLM.from_pretrained(
41-
'decapoda-research/llama-7b-hf,
41+
'decapoda-research/llama-7b-hf',
4242
device_map='auto',
4343
load_in_8bit=True,
4444
max_memory=f'{int(torch.cuda.mem_get_info()[0]/1024**3)-2}GB')
@@ -119,7 +119,7 @@ torch.nn.Embedding(...) -> bnb.nn.StableEmbedding(...) # recommended for NLP mo
119119
```
120120

121121
Note that by default all parameter tensors with less than 4096 elements are kept at 32-bit even if you initialize those parameters with 8-bit optimizers. This is done since such small tensors do not save much memory and often contain highly variable parameters (biases) or parameters that require high precision (batch norm, layer norm). You can change this behavior like so:
122-
```
122+
```python
123123
# parameter tensors with less than 16384 values are optimized in 32-bit
124124
# it is recommended to use multiplies of 4096
125125
adam = bnb.optim.Adam8bit(model.parameters(), min_8bit_size=16384)

0 commit comments

Comments
 (0)