You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _posts/2025-01-10-dev-experience.md
+43-21Lines changed: 43 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,34 +19,47 @@ The field of LLM inference is advancing at an unprecedented pace. With new model
19
19
20
20
We periodically release stable versions of vLLM to the [Python Package Index](https://pypi.org/project/vllm/), ensuring users can easily install them using standard Python package managers. For example:
21
21
22
-
| pip install vllm |
23
-
| :---- |
22
+
```sh
23
+
pip install vllm
24
+
```
24
25
25
26
For those who prefer a faster package manager, [**uv**](https://github.com/astral-sh/uv) has been gaining traction in the vLLM community. After setting up a Python environment with uv, installing vLLM is straightforward:
26
27
27
-
| uv pip install vllm |
28
-
| :---- |
28
+
```sh
29
+
uv pip install vllm
30
+
```
29
31
30
32
Refer to the [documentation](https://docs.vllm.ai/en/latest/getting_started/installation/gpu-cuda.html#install-released-versions) for more details on setting up [**uv**](https://github.com/astral-sh/uv). With a simple server-grade setup (Intel 8th Gen CPU), we can see [**uv**](https://github.com/astral-sh/uv) is 200x faster than pip:
31
33
32
-
|\# with cached packages, clean virtual environment$ time pip install vllm...pip install vllm 59.09s user 3.82s system 83% cpu 1:15.68 total \# with cached packages, clean virtual environment$ time uv pip install vllm....uv pip install vllm 0.17s user 0.57s system 193% cpu 0.383 total |
33
-
| :---- |
34
+
```sh
35
+
# with cached packages, clean virtual environment
36
+
$ time pip install vllm
37
+
...
38
+
pip install vllm 59.09s user 3.82s system 83% cpu 1:15.68 total
39
+
40
+
# with cached packages, clean virtual environment
41
+
$ time uv pip install vllm
42
+
...
43
+
uv pip install vllm 0.17s user 0.57s system 193% cpu 0.383 total
44
+
```
34
45
35
46
### Install the Latest vLLM from the Main Branch
36
47
37
48
To meet the community’s need for cutting-edge features and models, we provide nightly wheels for every commit on the main branch.
@@ -56,10 +69,13 @@ We understand that an active, engaged developer community is the backbone of inn
56
69
57
70
For Python developers who need to tweak and test vLLM’s Python code, there’s no need to compile kernels. Our solution allows you to get started in just a few minutes:
The VLLM\_USE\_PRECOMPILED=1 flag instructs the installer to use pre-compiled CUDA kernels instead of building them from source, significantly reducing installation time. This is perfect for developers focusing on Python-level features like API improvements, model support, or integration work.
78
+
The `VLLM_USE_PRECOMPILED=1` flag instructs the installer to use pre-compiled CUDA kernels instead of building them from source, significantly reducing installation time. This is perfect for developers focusing on Python-level features like API improvements, model support, or integration work.
63
79
64
80
This lightweight process runs efficiently, even on a laptop. For more advanced usage, please check the [documentation](https://docs.vllm.ai/en/latest/getting_started/installation/gpu-cuda.html#python-only-build-without-compilation).
65
81
@@ -69,23 +85,29 @@ For advanced contributors working with C++ code or CUDA kernels, we’ve optimiz
69
85
70
86
## Track Changes with Ease
71
87
72
-
The fast-evolving nature of LLM inference means interfaces and behaviors are still stabilizing. vLLM has been integrated into many workflows, including [OpenRLHF](https://github.com/OpenRLHF/OpenRLHF), [veRL](https://github.com/volcengine/verl), [open\_instruct](https://github.com/allenai/open-instruct), [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory), etc. We are working with them to stabilize interfaces and behaviors for LLM inference. To facilitate the process, we provide powerful tools for these powerful users to track changes across versions.
88
+
The fast-evolving nature of LLM inference means interfaces and behaviors are still stabilizing. vLLM has been integrated into many workflows, including [OpenRLHF](https://github.com/OpenRLHF/OpenRLHF), [veRL](https://github.com/volcengine/verl), [open_instruct](https://github.com/allenai/open-instruct), [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory), etc. We are working with them to stabilize interfaces and behaviors for LLM inference. To facilitate the process, we provide powerful tools for these powerful users to track changes across versions.
73
89
74
90
### Installing a Specific Commit
75
91
76
92
To simplify tracking and testing, we provide wheels for every commit in the main branch. Users can easily install any specific commit, which can be particularly useful to bisect and track the changes.
77
93
78
94
We recommend using [**uv**](https://github.com/astral-sh/uv) to install a specific commit:
79
95
80
-
|\# use full commit hash from the main branch export VLLM\_COMMIT=72d9c316d3f6ede485146fe5aabd4e61dbc59069uv pip install vllm \--extra-index-url https://wheels.vllm.ai/${VLLM\_COMMIT}|
In [**uv**](https://github.com/astral-sh/uv), packages in `--extra-index-url` have [higher priority than the default index](https://docs.astral.sh/uv/pip/compatibility/#packages-that-exist-on-multiple-indexes), which makes it possible to install a developing version prior to the latest public release (at the time of writing, it is v0.6.6.post1).
102
+
In [**uv**](https://github.com/astral-sh/uv), packages in `--extra-index-url` have [higher priority than the default index](https://docs.astral.sh/uv/pip/compatibility/#packages-that-exist-on-multiple-indexes), which makes it possible to install a developing version prior to the latest public release (at the time of writing, it is v0.6.6.post1).
84
103
85
-
In contrast, pip combines packages from \--extra-index-url and the default index, choosing only the latest version, which makes it difficult to install a developing version prior to the released version. Therefore, for pip users, it requires specifying a placeholder wheel name to install a specific commit:
104
+
In contrast, pip combines packages from `--extra-index-url` and the default index, choosing only the latest version, which makes it difficult to install a developing version prior to the released version. Therefore, for pip users, it requires specifying a placeholder wheel name to install a specific commit:
86
105
87
-
|\# use full commit hash from the main branch export VLLM\_COMMIT=33f460b17a54acb3b6cc0b03f4a17876cff5eafd pip install https://wheels.vllm.ai/${VLLM\_COMMIT}/vllm-1.0.0.dev-cp38-abi3-manylinux1\_x86\_64.whl|
0 commit comments