You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* Update Dockerfile
Update the Dockerfile to use Ubuntu 24.04 as the base image.
* Show the generated token count
* update README
Add links to the NPU models for the Qwen3.
---------
Co-authored-by: Xiake Sun <[email protected]>
@@ -725,7 +728,9 @@ Let's take [deepseek-ai/DeepSeek-R1-Distill-Qwen-7B](https://hf-mirror.com/deeps
725
728
Note:
726
729
727
730
1. The `ModelType "OpenVINO"` parameter is mandatory and must be explicitly set.
728
-
2. The `InferDevice` parameter is optional. If not specified, the system will prioritize using the GPU by default. If no GPU is available, it will automatically fall back to using the CPU. If InferDevice is explicitly set, the system will strictly use the specified device. If the specified device is unavailable, the system will follow the same fallback strategy as when InferDevice is not set (i.e., GPU first, then CPU).
731
+
2. The `InferDevice` parameter is optional:
732
+
- If not specified, the system will prioritize using the GPU by default. If no GPU is available, it will automatically fall back to using the CPU. If InferDevice is explicitly set, the system will strictly use the specified device. If the specified device is unavailable, the system will follow the same fallback strategy as when InferDevice is not set (i.e., GPU first, then CPU).
733
+
- If there are multiple GPUs in the environment, you can specify which GPU device to use by indicating GPU:<id>. For example, GPU:0 or GPU:1.
729
734
3. For more information on working with a Modelfile, see the [Modelfile](./docs/modelfile.md) documentation.
730
735
731
736
4. Unzip OpenVINO GenAI package and set environment
0 commit comments