You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Source the project-specific environment paths -> this will ensure the correct paths are set for the project and the TornadoVM SDK
150
+
# Expect to see: [INFO] Environment configured for Llama3 with TornadoVM at: C:\Users\YOUR_PATH_TO_TORNADOVM
151
+
set_paths.cmd
152
+
153
+
# Build the project using Maven (skip tests for faster build)
154
+
# mvn clean package -DskipTests or just make
155
+
make
156
+
157
+
# Run the model (make sure you have downloaded the model file first - see below)
158
+
python llama-tornado --gpu --verbose-init --opencl --model beehive-llama-3.2-1b-instruct-fp16.gguf --prompt "tell me a joke"
159
+
```
127
160
-----------
128
161
129
162
The above model can we swapped with one of the other models, such as `beehive-llama-3.2-3b-instruct-fp16.gguf` or `beehive-llama-3.2-8b-instruct-fp16.gguf`, depending on your needs.
@@ -182,7 +215,7 @@ Run a model with a text prompt:
182
215
#### GPU Execution (FP16 Model)
183
216
Enable GPU acceleration with Q8_0 quantization:
184
217
```bash
185
-
llama-tornado --gpu --verbose-init --model beehive-llama-3.2-1b-instruct-fp16.gguf --prompt "tell me a joke"
218
+
./llama-tornado --gpu --verbose-init --model beehive-llama-3.2-1b-instruct-fp16.gguf --prompt "tell me a joke"
0 commit comments