Skip to content

Commit fd841f1

Browse files
davidxiaepwalsh
authored andcommitted
[Docs] use uv in CPU installation docs (vllm-project#22089)
Signed-off-by: David Xia <[email protected]>
1 parent a26bcb0 commit fd841f1

File tree

3 files changed

+48
-31
lines changed

3 files changed

+48
-31
lines changed

docs/getting_started/installation/cpu/apple.inc.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# --8<-- [start:installation]
22

3-
vLLM has experimental support for macOS with Apple silicon. For now, users shall build from the source vLLM to natively run on macOS.
3+
vLLM has experimental support for macOS with Apple silicon. For now, users must build from source to natively run on macOS.
44

55
Currently the CPU implementation for macOS supports FP32 and FP16 datatypes.
66

@@ -23,20 +23,20 @@ Currently the CPU implementation for macOS supports FP32 and FP16 datatypes.
2323
# --8<-- [end:pre-built-wheels]
2424
# --8<-- [start:build-wheel-from-source]
2525

26-
After installation of XCode and the Command Line Tools, which include Apple Clang, execute the following commands to build and install vLLM from the source.
26+
After installation of XCode and the Command Line Tools, which include Apple Clang, execute the following commands to build and install vLLM from source.
2727

2828
```bash
2929
git clone https://github.com/vllm-project/vllm.git
3030
cd vllm
31-
pip install -r requirements/cpu.txt
32-
pip install -e .
31+
uv pip install -r requirements/cpu.txt
32+
uv pip install -e .
3333
```
3434

3535
!!! note
36-
On macOS the `VLLM_TARGET_DEVICE` is automatically set to `cpu`, which currently is the only supported device.
36+
On macOS the `VLLM_TARGET_DEVICE` is automatically set to `cpu`, which is currently the only supported device.
3737

3838
!!! example "Troubleshooting"
39-
If the build has error like the following snippet where standard C++ headers cannot be found, try to remove and reinstall your
39+
If the build fails with errors like the following where standard C++ headers cannot be found, try to remove and reinstall your
4040
[Command Line Tools for Xcode](https://developer.apple.com/download/all/).
4141

4242
```text

docs/getting_started/installation/cpu/build.inc.md

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,33 +1,39 @@
1-
First, install recommended compiler. We recommend to use `gcc/g++ >= 12.3.0` as the default compiler to avoid potential problems. For example, on Ubuntu 22.4, you can run:
1+
First, install the recommended compiler. We recommend using `gcc/g++ >= 12.3.0` as the default compiler to avoid potential problems. For example, on Ubuntu 22.4, you can run:
22

33
```bash
44
sudo apt-get update -y
55
sudo apt-get install -y --no-install-recommends ccache git curl wget ca-certificates gcc-12 g++-12 libtcmalloc-minimal4 libnuma-dev ffmpeg libsm6 libxext6 libgl1 jq lsof
66
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 10 --slave /usr/bin/g++ g++ /usr/bin/g++-12
77
```
88

9-
Second, clone vLLM project:
9+
Second, clone the vLLM project:
1010

1111
```bash
1212
git clone https://github.com/vllm-project/vllm.git vllm_source
1313
cd vllm_source
1414
```
1515

16-
Third, install Python packages for vLLM CPU backend building:
16+
Third, install required dependencies:
1717

1818
```bash
19-
pip install --upgrade pip
20-
pip install -v -r requirements/cpu-build.txt --extra-index-url https://download.pytorch.org/whl/cpu
21-
pip install -v -r requirements/cpu.txt --extra-index-url https://download.pytorch.org/whl/cpu
19+
uv pip install -r requirements/cpu-build.txt --torch-backend auto
20+
uv pip install -r requirements/cpu.txt --torch-backend auto
2221
```
2322

24-
Finally, build and install vLLM CPU backend:
23+
??? console "pip"
24+
```bash
25+
pip install --upgrade pip
26+
pip install -v -r requirements/cpu-build.txt --extra-index-url https://download.pytorch.org/whl/cpu
27+
pip install -v -r requirements/cpu.txt --extra-index-url https://download.pytorch.org/whl/cpu
28+
```
29+
30+
Finally, build and install vLLM:
2531

2632
```bash
2733
VLLM_TARGET_DEVICE=cpu python setup.py install
2834
```
2935

30-
If you want to develop vllm, install it in editable mode instead.
36+
If you want to develop vLLM, install it in editable mode instead.
3137

3238
```bash
3339
VLLM_TARGET_DEVICE=cpu python setup.py develop

docs/getting_started/installation/cpu/s390x.inc.md

Lines changed: 28 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# --8<-- [start:installation]
22

3-
vLLM has experimental support for s390x architecture on IBM Z platform. For now, users shall build from the vLLM source to natively run on IBM Z platform.
3+
vLLM has experimental support for s390x architecture on IBM Z platform. For now, users must build from source to natively run on IBM Z platform.
44

55
Currently the CPU implementation for s390x architecture supports FP32 datatype only.
66

@@ -40,21 +40,32 @@ curl https://sh.rustup.rs -sSf | sh -s -- -y && \
4040
. "$HOME/.cargo/env"
4141
```
4242

43-
Execute the following commands to build and install vLLM from the source.
43+
Execute the following commands to build and install vLLM from source.
4444

4545
!!! tip
46-
Please build the following dependencies, `torchvision`, `pyarrow` from the source before building vLLM.
46+
Please build the following dependencies, `torchvision`, `pyarrow` from source before building vLLM.
4747

4848
```bash
4949
sed -i '/^torch/d' requirements-build.txt # remove torch from requirements-build.txt since we use nightly builds
50-
pip install -v \
51-
--extra-index-url https://download.pytorch.org/whl/nightly/cpu \
50+
uv pip install -v \
51+
--torch-backend auto \
5252
-r requirements-build.txt \
5353
-r requirements-cpu.txt \
5454
VLLM_TARGET_DEVICE=cpu python setup.py bdist_wheel && \
55-
pip install dist/*.whl
55+
uv pip install dist/*.whl
5656
```
5757

58+
??? console "pip"
59+
```bash
60+
sed -i '/^torch/d' requirements-build.txt # remove torch from requirements-build.txt since we use nightly builds
61+
pip install -v \
62+
--extra-index-url https://download.pytorch.org/whl/nightly/cpu \
63+
-r requirements-build.txt \
64+
-r requirements-cpu.txt \
65+
VLLM_TARGET_DEVICE=cpu python setup.py bdist_wheel && \
66+
pip install dist/*.whl
67+
```
68+
5869
# --8<-- [end:build-wheel-from-source]
5970
# --8<-- [start:pre-built-images]
6071

@@ -63,19 +74,19 @@ Execute the following commands to build and install vLLM from the source.
6374

6475
```bash
6576
docker build -f docker/Dockerfile.s390x \
66-
--tag vllm-cpu-env .
77+
--tag vllm-cpu-env .
6778

68-
# Launching OpenAI server
79+
# Launch OpenAI server
6980
docker run --rm \
70-
--privileged=true \
71-
--shm-size=4g \
72-
-p 8000:8000 \
73-
-e VLLM_CPU_KVCACHE_SPACE=<KV cache space> \
74-
-e VLLM_CPU_OMP_THREADS_BIND=<CPU cores for inference> \
75-
vllm-cpu-env \
76-
--model=meta-llama/Llama-3.2-1B-Instruct \
77-
--dtype=float \
78-
other vLLM OpenAI server arguments
81+
--privileged true \
82+
--shm-size 4g \
83+
-p 8000:8000 \
84+
-e VLLM_CPU_KVCACHE_SPACE=<KV cache space> \
85+
-e VLLM_CPU_OMP_THREADS_BIND=<CPU cores for inference> \
86+
vllm-cpu-env \
87+
--model meta-llama/Llama-3.2-1B-Instruct \
88+
--dtype float \
89+
other vLLM OpenAI server arguments
7990
```
8091

8192
# --8<-- [end:build-image-from-source]

0 commit comments

Comments
 (0)