Skip to content

Commit 05902d1

Browse files
authored
does not support setting top_k (#958)
* Improve interface compatibility Add Dockerfile for Linux platform * del unimportant comments * Update README.md * add DeepSeek-R1-Distill-Qwen-14B-int4-ov * Does not support setting top_k update DeepSeek model Modelfile
1 parent 04488ee commit 05902d1

File tree

4 files changed

+16
-33
lines changed

4 files changed

+16
-33
lines changed

modules/ollama_openvino/README.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -584,17 +584,17 @@ Getting started with large language models and using the [GenAI](https://github.
584584
We provide two ways to download the executable file of Ollama, one is to download it from Google Drive, and the other is to download it from Baidu Drive:
585585
## Google Driver
586586
### Windows
587-
[Download exe](https://drive.google.com/file/d/1iizO9iLhSJGFUu6BgY3EwOchrCyzImUN/view?usp=drive_link) + [Download OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250320/openvino_genai_windows_2025.2.0.0.dev20250320_x86_64.zip)
587+
[Download exe](https://drive.google.com/file/d/1Sep1IdGn7mJaE8PCXKYxp_aj1ljiPvpN/view?usp=sharing) + [Download OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250320/openvino_genai_windows_2025.2.0.0.dev20250320_x86_64.zip)
588588

589589
### Linux(Ubuntu 22.04)
590-
[Download](https://drive.google.com/file/d/1HEyZNNCbWSidKNQl4MRsD8FuwEZtdyew/view?usp=drive_link) + [Donwload OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250320/openvino_genai_ubuntu22_2025.2.0.0.dev20250320_x86_64.tar.gz)
590+
[Download](https://drive.google.com/file/d/1DdBoEGp_eoyJPbpMGVbEivihYSKrCMGt/view?usp=sharing) + [Donwload OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250320/openvino_genai_ubuntu22_2025.2.0.0.dev20250320_x86_64.tar.gz)
591591

592592
## 百度云盘
593593
### Windows
594-
[Download exe](https://pan.baidu.com/s/10SQLKaGOYIR3NrIhig1xgQ?pwd=8v8w) + [Download OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250320/openvino_genai_windows_2025.2.0.0.dev20250320_x86_64.zip)
594+
[Download exe](https://pan.baidu.com/s/1TCH7rYSPr8jQDHLvCeXdLw?pwd=6bk9) + [Download OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250320/openvino_genai_windows_2025.2.0.0.dev20250320_x86_64.zip)
595595

596596
### Linux(Ubuntu 22.04)
597-
[Download](https://pan.baidu.com/s/1WhzcKRQY_KVLZhy5oa0UIA?pwd=zxgx) + [Donwload OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250320/openvino_genai_ubuntu22_2025.2.0.0.dev20250320_x86_64.tar.gz)
597+
[Download](https://pan.baidu.com/s/1UVO0ZK4DFTjTwfarQ8LUIw?pwd=pxkd) + [Donwload OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250320/openvino_genai_ubuntu22_2025.2.0.0.dev20250320_x86_64.tar.gz)
598598

599599
## Docker
600600
### Linux
@@ -659,19 +659,19 @@ We added two new parameters to Modelfile based on the original parameters:
659659

660660
### Start Ollama
661661

662-
1. First, set GODEGUG=cgocheck=0 env:
662+
1. First, set GODEBUG=cgocheck=0 env:
663663

664664
#### Linux
665665
```shell
666-
export GODEGUG=cgocheck=0
666+
export GODEBUG=cgocheck=0
667667
```
668668

669669
#### Windows
670670
```shell
671671
set GODEBUG=cgocheck=0
672672
```
673673

674-
2. Next, `ollama serve` is used when you want to start ollama (You must use the Ollama compiled by ollama_ov to start the serve):
674+
2. Next, `ollama serve` is used when you want to start ollama (you must use the Ollama compiled by ollama_ov to start the serve):
675675

676676
```shell
677677
ollama serve
@@ -861,11 +861,11 @@ Then build and run Ollama from the root directory of the repository:
861861

862862
### Running local builds
863863

864-
1. First, set GODEGUG=cgocheck=0 env:
864+
1. First, set GODEBUG=cgocheck=0 env:
865865

866866
#### Linux
867867
```shell
868-
export GODEGUG=cgocheck=0
868+
export GODEBUG=cgocheck=0
869869
```
870870

871871
#### Windows
Lines changed: 3 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,6 @@
11
FROM DeepSeek-R1-Distill-Llama-8B-int4-ov.tar.gz
22
ModelType "OpenVINO"
33
InferDevice "GPU"
4-
PARAMETER stop "<|end▁of▁sentence|>"
5-
PARAMETER stop "<|User|>"
6-
PARAMETER stop "</User|>"
7-
PARAMETER stop "<|end_of_sentence|>"
8-
PARAMETER stop "</|"
9-
PARAMETER max_new_token 4096
10-
PARAMETER stop_id 128001
11-
PARAMETER repeat_penalty 1.5
12-
PARAMETER top_p 0.95
13-
PARAMETER top_k 50
14-
PARAMETER temperature 0.8
4+
PARAMETER repeat_penalty 1.0
5+
PARAMETER top_p 1.0
6+
PARAMETER temperature 1.0
Lines changed: 3 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,6 @@
11
FROM DeepSeek-R1-Distill-Qwen-7B-int4-ov.tar.gz
22
ModelType "OpenVINO"
33
InferDevice "GPU"
4-
PARAMETER stop "<|end▁of▁sentence|>"
5-
PARAMETER stop "<|User|>"
6-
PARAMETER stop "</User|>"
7-
PARAMETER stop "<|end_of_sentence|>"
8-
PARAMETER stop "</|"
9-
PARAMETER max_new_token 4096
10-
PARAMETER stop_id 151643
11-
PARAMETER stop_id 151647
12-
PARAMETER repeat_penalty 1.5
13-
PARAMETER top_p 0.95
14-
PARAMETER top_k 50
15-
PARAMETER temperature 0.8
4+
PARAMETER repeat_penalty 1.0
5+
PARAMETER top_p 1.0
6+
PARAMETER temperature 1.0

modules/ollama_openvino/genai/genai.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -167,7 +167,7 @@ func SetSamplingParams(samplingparameters *SamplingParams) *C.ov_genai_generatio
167167
C.ov_genai_generation_config_set_max_new_tokens(cConfig, C.size_t(samplingparameters.MaxNewToken))
168168
C.ov_genai_generation_config_set_temperature(cConfig, C.float(samplingparameters.Temp))
169169
C.ov_genai_generation_config_set_top_p(cConfig, C.float(samplingparameters.TopP))
170-
C.ov_genai_generation_config_set_top_k(cConfig, C.size_t(samplingparameters.TopK))
170+
// C.ov_genai_generation_config_set_top_k(cConfig, C.size_t(samplingparameters.TopK))
171171

172172
if samplingparameters.StopIds != nil {
173173
cStopArray := C.malloc(C.size_t(len(samplingparameters.StopIds)) * C.size_t(unsafe.Sizeof(C.int64_t(0))))

0 commit comments

Comments
 (0)