You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
You can refer to the general [*Prepare and Quantize*](README.md#prepare-and-quantize) guide for model prepration, or download an already quantized model like [Meta-Llama-3-8B-Instruct-Q4_0.gguf](https://huggingface.co/aptha/Meta-Llama-3-8B-Instruct-Q4_0-GGUF/resolve/main/Meta-Llama-3-8B-Instruct-Q4_0.gguf).
356
+
You can refer to the general [*Prepare and Quantize*](README.md#prepare-and-quantize) guide for model preparation, or download an already quantized model like[llama-2-7b.Q4_0.gguf](https://huggingface.co/TheBloke/Llama-2-7B-GGUF/blob/main/llama-2-7b.Q4_0.gguf) or[Meta-Llama-3-8B-Instruct-Q4_0.gguf](https://huggingface.co/aptha/Meta-Llama-3-8B-Instruct-Q4_0-GGUF/resolve/main/Meta-Llama-3-8B-Instruct-Q4_0.gguf).
357
357
358
358
##### Check device
359
359
@@ -399,12 +399,16 @@ Choose one of following methods to run.
399
399
- Use device 0:
400
400
401
401
```sh
402
-
./examples/sycl/run-llama.sh 0
402
+
./examples/sycl/run-llama2.sh 0
403
+
# OR
404
+
./examples/sycl/run-llama3.sh 0
403
405
```
404
406
- Use multiple devices:
405
407
406
408
```sh
407
-
./examples/sycl/run-llama.sh
409
+
./examples/sycl/run-llama2.sh
410
+
# OR
411
+
./examples/sycl/run-llama3.sh
408
412
```
409
413
410
414
2. Command line
@@ -427,13 +431,13 @@ Examples:
427
431
- Use device 0:
428
432
429
433
```sh
430
-
ZES_ENABLE_SYSMAN=1 ./build/bin/llama-cli -no-cnv -m models/Meta-Llama-3-8B-Instruct-Q4_0.gguf -p "Building a website can be done in 10 simple steps:" -n 400 -e -ngl 33 -sm none -mg 0
434
+
ZES_ENABLE_SYSMAN=1 ./build/bin/llama-cli -no-cnv -m models/llama-2-7b.Q4_0.gguf -p "Building a website can be done in 10 simple steps:" -n 400 -e -ngl 99 -sm none -mg 0
431
435
```
432
436
433
437
- Use multiple devices:
434
438
435
439
```sh
436
-
ZES_ENABLE_SYSMAN=1 ./build/bin/llama-cli -no-cnv -m models/Meta-Llama-3-8B-Instruct-Q4_0.gguf -p "Building a website can be done in 10 simple steps:" -n 400 -e -ngl 33 -sm layer
440
+
ZES_ENABLE_SYSMAN=1 ./build/bin/llama-cli -no-cnv -m models/llama-2-7b.Q4_0.gguf -p "Building a website can be done in 10 simple steps:" -n 400 -e -ngl 99 -sm layer
437
441
```
438
442
439
443
*Notes:*
@@ -454,7 +458,7 @@ use 1 SYCL GPUs: [0] with Max compute units:512
454
458
455
459
1. Install GPU driver
456
460
457
-
Intel GPU drivers instructions guide and download page can be found here: [Get intel GPU Drivers](https://www.intel.com/content/www/us/en/products/docs/discrete-gpus/arc/software/drivers.html).
461
+
Intel GPU drivers instructions guide and download page can be found here: [Get Intel GPU Drivers](https://www.intel.com/content/www/us/en/products/docs/discrete-gpus/arc/software/drivers.html).
458
462
459
463
2. Install Visual Studio
460
464
@@ -631,7 +635,7 @@ Once it is completed, final results will be in **build/Release/bin**
631
635
632
636
#### Retrieve and prepare model
633
637
634
-
You can refer to the general [*Prepare and Quantize*](README.md#prepare-and-quantize) guide for model prepration, or simply download [Meta-Llama-3-8B-Instruct-Q4_0.gguf](https://huggingface.co/aptha/Meta-Llama-3-8B-Instruct-Q4_0-GGUF/blob/main/Meta-Llama-3-8B-Instruct-Q4_0.gguf) model as example.
638
+
You can refer to the general [*Prepare and Quantize*](README.md#prepare-and-quantize) guide for model preparation, or download an already quantized model like [llama-2-7b.Q4_0.gguf](https://huggingface.co/TheBloke/Llama-2-7B-GGUF/blob/main/llama-2-7b.Q4_0.gguf) or [Meta-Llama-3-8B-Instruct-Q4_0.gguf](https://huggingface.co/aptha/Meta-Llama-3-8B-Instruct-Q4_0-GGUF/resolve/main/Meta-Llama-3-8B-Instruct-Q4_0.gguf).
635
639
636
640
##### Check device
637
641
@@ -676,7 +680,13 @@ Choose one of following methods to run.
676
680
1. Script
677
681
678
682
```
679
-
examples\sycl\win-run-llama.bat
683
+
examples\sycl\win-run-llama-2.bat
684
+
```
685
+
686
+
or
687
+
688
+
```
689
+
examples\sycl\win-run-llama-3.bat
680
690
```
681
691
682
692
2. Command line
@@ -700,13 +710,13 @@ Examples:
700
710
- Use device 0:
701
711
702
712
```
703
-
build\bin\llama-cli.exe -no-cnv -m models\Meta-Llama-3-8B-Instruct-Q4_0.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 400 -e -ngl 99 -sm none -mg 0
713
+
build\bin\llama-cli.exe -no-cnv -m models\llama-2-7b.Q4_0.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 400 -e -ngl 99 -sm none -mg 0
704
714
```
705
715
706
716
- Use multiple devices:
707
717
708
718
```
709
-
build\bin\llama-cli.exe -no-cnv -m models\Meta-Llama-3-8B-Instruct-Q4_0.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 400 -e -ngl 99 -sm layer
719
+
build\bin\llama-cli.exe -no-cnv -m models\llama-2-7b.Q4_0.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 400 -e -ngl 99 -sm layer
0 commit comments