Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion docker/Dockerfile-NPU
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Set to other image if needed
FROM quay.io/ascend/vllm-ascend:v0.9.2rc1
FROM quay.io/ascend/vllm-ascend:v0.9.2rc1-openeuler

ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"

Expand All @@ -14,4 +14,12 @@ RUN export PLATFORM="ascend" && \
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/Ascend/ascend-toolkit/latest/`uname -i`-linux/devlib && \
pip install -v -e /workspace/unified-cache-management --no-build-isolation

# Apply patch for vLLM
RUN cd /vllm-workspace/vllm \
&& git apply /workspace/unified-cache-management/ucm/integration/vllm/patch/0.9.2/vllm-adapt.patch

# Apply patch for vLLM-ascend
RUN cd /vllm-workspace/vllm-ascend \
&& git apply /workspace/unified-cache-management/ucm/integration/vllm/patch/0.9.2/vllm-ascend-adapt.patch

CMD ["/bin/bash"]
8 changes: 8 additions & 0 deletions docs/source/getting-started/quickstart_vllm_ascend.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,14 @@ Download the pre-built `vllm-ascend` docker image and build unified-cache-manage
cd unified-cache-management
docker build -t ucm-vllm:latest -f ./docker/Dockerfile-NPU ./
```
vllm-ascend provides two variants: **Ubuntu** and **openEuler**.
The `Dockerfile-NPU` uses the **openEuler** variant by default.

If you want to use the **Ubuntu** variant, please remove the `-openeuler` suffix and use the following image instead:

```text
quay.io/ascend/vllm-ascend:v0.9.2rc1
```
Then run your container using following command. You can add or remove Docker parameters as needed.
```bash
# Update DEVICE according to your device (/dev/davinci[0-7])
Expand Down