Skip to content

Commit fa90a98

Browse files
saood06ngxson
andauthored
Docs update (#509)
* use npm as deps manager and vite as bundler * update XTC docs --------- Co-authored-by: Xuan Son Nguyen <[email protected]>
1 parent 58f08e4 commit fa90a98

File tree

2 files changed

+35
-6
lines changed

2 files changed

+35
-6
lines changed

examples/main/README.md

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -241,13 +241,11 @@ Example usage: `--mirostat 2 --mirostat-lr 0.05 --mirostat-ent 3.0`
241241

242242
### XTC Sampling (Exclude Top Choices)
243243

244-
The function of this sampler is conrolled by `--xtc-probability` and `--xtc-threshold`. `--xtc-probability` takes values between
245-
0 and 1 (<=0 turns this sampler off) and defines the probability for randomly invoking the sampler. `--xtc-threshold`
246-
defines the token probability threshold. Tokens with probability greater than this threshold will be excluded from the sampling.
247-
The sampler is turned off for `threshold > 0.5`.
244+
- --xtc-probability p: xtc probability (default: 0.0, maximum: 1.0) => (p*100)% chance to invoke the sampler.
245+
- --xtc-threshold t : xtc threshold (default: 1.0 minimum : 0.0) => (t*100)% is the lower-bound for what probability is needed for a token to be considered a "Top choice", and only the lowest probability top choice is kept, all others are removed.
246+
247+
The sampler is turned off for `threshold > 0.5`, and `--xtc-probability <=0`.
248248

249-
- --xtc-probability p: xtc probability (default: 0.0 => disabled)
250-
- --xtc-threshold t : xtc threshold (default: 1.0 => disabled)
251249

252250
### Top-n-sigma Sampling
253251

examples/server/README.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -290,6 +290,37 @@ logging:
290290
cmake --build build --config Release -t llama-server
291291
```
292292

293+
## Web UI
294+
295+
The project includes a web-based user interface that enables interaction with the model through the `/chat/completions` endpoint.
296+
297+
The web UI is developed using:
298+
- `vue` framework for frontend development
299+
- `tailwindcss` and `daisyui` for styling
300+
- `vite` for build tooling
301+
302+
A pre-built version is available as a single HTML file under `/public` directory.
303+
304+
To build or to run the dev server (with hot reload):
305+
306+
```sh
307+
# make sure you have nodejs installed
308+
cd examples/server/webui
309+
npm i
310+
311+
# to run the dev server
312+
npm run dev
313+
314+
# to build the public/index.html
315+
npm run build
316+
```
317+
318+
NOTE: if you are using the vite dev server, you can change the API base URL to llama.cpp. To do that, run this code snippet in browser's console:
319+
320+
```js
321+
localStorage.setItem('base', 'http://localhost:8080')
322+
```
323+
293324
## Quick Start
294325

295326
To get started right away, run the following command, making sure to use the correct path for the model you have:

0 commit comments

Comments
 (0)