Skip to content

Commit 8fb681b

Browse files
committed
updated readme
1 parent bee4c7c commit 8fb681b

File tree

1 file changed

+9
-1
lines changed

1 file changed

+9
-1
lines changed

examples/main/README.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -265,7 +265,15 @@ Being experimental and unique, XTC is disabled by default. The recommended combi
265265

266266
Example usage: `--xtc-probability 0.5 --xtc-threshold 0.1`
267267

268-
### Logit Bias
268+
### Top-nσ Sampling
269+
270+
- `--top-nsigma N`: Limit the next token selection to a subset of tokens with pre-softmax logits that are within n * σ less than the max logit (default: -1, -1 = disabled).
271+
272+
Top-nσ sampling is a text generation method that selects tokens based on a statistical threshold in pre-softmax logits. It works by only sampling from tokens with logits that are within n * σ of the maximum logit. This method helps maintain a stable sampling space regardless of temperature scaling, allowing it to perform well on reasoning tasks even in high temperatures. Without complex probability manipulation, it efficiently filters tokens directly on the pre-softmax logits. A higher value for top-nsigma (e.g., 5) will take more noisy tokens into consideration, while a lower value (e.g., 1) will focous on the more informative region of the sampling space.
273+
274+
Example usage: `--top-nsigma 1`
275+
276+
### Logit Bias
269277

270278
- `-l TOKEN_ID(+/-)BIAS, --logit-bias TOKEN_ID(+/-)BIAS`: Modify the likelihood of a token appearing in the generated text completion.
271279

0 commit comments

Comments
 (0)