Skip to content

Commit 4b60f71

Browse files
authored
Fix npu docker run command in assistant demo (openvinotoolkit#3226)
1 parent c9658a3 commit 4b60f71

File tree

1 file changed

+5
-4
lines changed

1 file changed

+5
-4
lines changed

demos/code_local_assistant/README.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ Since we do not want to wait for the code to appear, we need to use smaller mode
3333
Code completion works in non-streaming, unary mode. Do not use instruct model, there is no chat involved in the process.
3434

3535
Export `Qwen/Qwen2.5-Coder-1.5B`:
36-
```baconsolesh
36+
```console
3737
python export_model.py text_generation --source_model Qwen/Qwen2.5-Coder-1.5B --weight-format int4 --config_file_path models/config_all.json --model_repository_path models --target_device NPU --overwrite_models
3838
```
3939

@@ -54,7 +54,7 @@ Examine that workspace is set up properly `models/config_all.json`:
5454
}
5555
```
5656

57-
```console
57+
```bash
5858
tree models
5959
models
6060
├── codellama
@@ -100,13 +100,14 @@ Run OpenVINO Model Server with both models loaded at the same time:
100100
### Windows: deploying on bare metal
101101
Please refer to OpenVINO Model Server installation first: [link](../../docs/deploying_server_baremetal.md)
102102

103-
```console
103+
```bat
104104
ovms --rest_port 8000 --config_path ./models/config_all.json
105105
```
106106

107107
### Linux: via Docker
108108
```bash
109-
docker run -d --rm -v $(pwd)/:/workspace/ -p 8000:8000 openvino/model_server:2025.1 --rest_port 8000 --config_path /workspace/models/config_all.json
109+
docker run -d --rm --device /dev/accel --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -u $(id -u):$(id -g) \
110+
-p 8000:8000 -v $(pwd)/:/workspace/ openvino/model_server:2025.1 --rest_port 8000 --config_path /workspace/models/config_all.json
110111
```
111112

112113
## Set Up Visual Studio Code

0 commit comments

Comments
 (0)