File tree Expand file tree Collapse file tree 1 file changed +3
-3
lines changed
Expand file tree Collapse file tree 1 file changed +3
-3
lines changed Original file line number Diff line number Diff line change @@ -50,13 +50,13 @@ python3 launch.py llama3_2_3b_instruct_q40 # Downloads the model to the root dev
50505 . Start workers on all ** 🔹 WORKER** devices:
5151
5252``` sh
53- sudo nice -n -20 ./dllama worker --port 9999 --nthreads 4
53+ ./dllama worker --port 9999 --nthreads 4
5454```
5555
56566 . Run the inference to test if everything works fine on the ** 🔸 ROOT** device:
5757
5858``` sh
59- sudo nice -n -20 ./dllama inference \
59+ ./dllama inference \
6060 --prompt " Hello world" \
6161 --steps 32 \
6262 --model models/llama3_2_3b_instruct_q40/dllama_model_llama3_2_3b_instruct_q40.m \
@@ -70,7 +70,7 @@ sudo nice -n -20 ./dllama inference \
70707 . To run the API server, start it on the ** 🔸 ROOT** device:
7171
7272``` sh
73- sudo nice -n -20 ./dllama-api \
73+ ./dllama-api \
7474 --port 9999 \
7575 --model models/llama3_2_3b_instruct_q40/dllama_model_llama3_2_3b_instruct_q40.m \
7676 --tokenizer models/llama3_2_3b_instruct_q40/dllama_tokenizer_llama3_2_3b_instruct_q40.t \
You can’t perform that action at this time.
0 commit comments