File tree Expand file tree Collapse file tree 1 file changed +5
-4
lines changed
demos/code_local_assistant Expand file tree Collapse file tree 1 file changed +5
-4
lines changed Original file line number Diff line number Diff line change @@ -33,7 +33,7 @@ Since we do not want to wait for the code to appear, we need to use smaller mode
33
33
Code completion works in non-streaming, unary mode. Do not use instruct model, there is no chat involved in the process.
34
34
35
35
Export ` Qwen/Qwen2.5-Coder-1.5B ` :
36
- ``` baconsolesh
36
+ ``` console
37
37
python export_model.py text_generation --source_model Qwen/Qwen2.5-Coder-1.5B --weight-format int4 --config_file_path models/config_all.json --model_repository_path models --target_device NPU --overwrite_models
38
38
```
39
39
@@ -54,7 +54,7 @@ Examine that workspace is set up properly `models/config_all.json`:
54
54
}
55
55
```
56
56
57
- ``` console
57
+ ``` bash
58
58
tree models
59
59
models
60
60
├── codellama
@@ -100,13 +100,14 @@ Run OpenVINO Model Server with both models loaded at the same time:
100
100
### Windows: deploying on bare metal
101
101
Please refer to OpenVINO Model Server installation first: [ link] ( ../../docs/deploying_server_baremetal.md )
102
102
103
- ``` console
103
+ ``` bat
104
104
ovms --rest_port 8000 --config_path ./models/config_all.json
105
105
```
106
106
107
107
### Linux: via Docker
108
108
``` bash
109
- docker run -d --rm -v $( pwd) /:/workspace/ -p 8000:8000 openvino/model_server:2025.1 --rest_port 8000 --config_path /workspace/models/config_all.json
109
+ docker run -d --rm --device /dev/accel --group-add=$( stat -c " %g" /dev/dri/render* | head -n 1) -u $( id -u) :$( id -g) \
110
+ -p 8000:8000 -v $( pwd) /:/workspace/ openvino/model_server:2025.1 --rest_port 8000 --config_path /workspace/models/config_all.json
110
111
```
111
112
112
113
## Set Up Visual Studio Code
You can’t perform that action at this time.
0 commit comments