Skip to content

Commit bc13f7c

Browse files
authored
[Doc] Update installation instructions for vllm 0.16.0 (#1505)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
1 parent 1f862ce commit bc13f7c

File tree

2 files changed

+2
-26
lines changed

2 files changed

+2
-26
lines changed

docs/getting_started/installation/gpu/cuda.inc.md

Lines changed: 1 addition & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -20,20 +20,7 @@ Therefore, it is recommended to install vLLM and vLLM-Omni with a **fresh new**
2020

2121
vLLM-Omni is built based on vLLM. Please install it with command below.
2222
```bash
23-
# vllm 0.16.0 is still under prerelease
24-
uv pip install --prerelease=allow vllm --extra-index-url https://wheels.vllm.ai/2d5be1dd5ce2e44dfea53ea03ff61143da5137eb
25-
26-
# vllm 0.16.0 may have some bugs for cuda 12.9, here is how we solve them:
27-
export FLASHINFER_CUDA_TAG="$(python3 -c 'import torch; print((torch.version.cuda or "12.4").replace(".", ""))')"
28-
29-
uv pip install --upgrade --force-reinstall \
30-
"flashinfer-python==0.6.3" \
31-
"flashinfer-cubin==0.6.3" \
32-
"flashinfer-jit-cache==0.6.3" \
33-
--extra-index-url "https://flashinfer.ai/whl/cu${FLASHINFER_CUDA_TAG}"
34-
35-
uv pip install --upgrade --force-reinstall "nvidia-cublas-cu12==12.9.1.4"
36-
uv pip install --upgrade --force-reinstall "numpy==2.2.6"
23+
uv pip install vllm --torch-backend=auto
3724
```
3825

3926
#### Installation of vLLM-Omni

docs/getting_started/quickstart.md

Lines changed: 1 addition & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -19,18 +19,7 @@ uv venv --python 3.12 --seed
1919
source .venv/bin/activate
2020

2121
# On CUDA
22-
# vllm 0.16.0 is still under prerelease
23-
uv pip install --prerelease=allow vllm --extra-index-url https://wheels.vllm.ai/2d5be1dd5ce2e44dfea53ea03ff61143da5137eb
24-
# vllm 0.16.0 may have some bugs for cuda 12.9, here is how we solve them:
25-
export FLASHINFER_CUDA_TAG="$(python3 -c 'import torch; print((torch.version.cuda or "12.4").replace(".", ""))')"
26-
uv pip install --upgrade --force-reinstall \
27-
"flashinfer-python==0.6.3" \
28-
"flashinfer-cubin==0.6.3" \
29-
"flashinfer-jit-cache==0.6.3" \
30-
--extra-index-url "https://flashinfer.ai/whl/cu${FLASHINFER_CUDA_TAG}"
31-
uv pip install --upgrade --force-reinstall "nvidia-cublas-cu12==12.9.1.4"
32-
uv pip install --upgrade --force-reinstall "numpy==2.2.6"
33-
22+
uv pip install vllm==0.16.0 --torch-backend=auto
3423

3524
# On ROCm
3625
uv pip install vllm==0.16.0 --extra-index-url https://wheels.vllm.ai/rocm/0.16.0/rocm700

0 commit comments

Comments
 (0)