Skip to content

Commit ca7b7fa

Browse files
committed
add support for deepseek distilled llama 8B model
1 parent d134211 commit ca7b7fa

File tree

2 files changed

+2
-2
lines changed

2 files changed

+2
-2
lines changed

.DS_Store

0 Bytes
Binary file not shown.

ray-services/ray-service-llama-3.2-CPU-LLAMA-Graviton.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,8 @@ spec:
2929
pip: ["llama_cpp_python", "transformers==4.46.0"]
3030
env_vars:
3131
LD_LIBRARY_PATH: "/home/ray/anaconda3/lib:$LD_LIBRARY_PATH"
32-
MODEL_ID: "bartowski/Llama-3.2-3B-Instruct-GGUF"
33-
MODEL_FILENAME: "Llama-3.2-3B-Instruct-Q4_0.gguf"
32+
MODEL_ID: "bartowski/DeepSeek-R1-Distill-Llama-8B-GGUF"
33+
MODEL_FILENAME: "DeepSeek-R1-Distill-Llama-8B-Q4_0.gguf"
3434
N_CTX: "0"
3535
N_THREADS : "32"
3636
FORCE_CMAKE: "1"

0 commit comments

Comments
 (0)