Skip to content

Commit a16c6d7

Browse files
committed
fix format
Signed-off-by: youkaichao <[email protected]>
1 parent a6df2f4 commit a16c6d7

File tree

1 file changed

+43
-21
lines changed

1 file changed

+43
-21
lines changed

_posts/2025-01-10-dev-experience.md

Lines changed: 43 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -19,34 +19,47 @@ The field of LLM inference is advancing at an unprecedented pace. With new model
1919

2020
We periodically release stable versions of vLLM to the [Python Package Index](https://pypi.org/project/vllm/), ensuring users can easily install them using standard Python package managers. For example:
2121

22-
| pip install vllm |
23-
| :---- |
22+
```sh
23+
pip install vllm
24+
```
2425

2526
For those who prefer a faster package manager, [**uv**](https://github.com/astral-sh/uv) has been gaining traction in the vLLM community. After setting up a Python environment with uv, installing vLLM is straightforward:
2627

27-
| uv pip install vllm |
28-
| :---- |
28+
```sh
29+
uv pip install vllm
30+
```
2931

3032
Refer to the [documentation](https://docs.vllm.ai/en/latest/getting_started/installation/gpu-cuda.html#install-released-versions) for more details on setting up [**uv**](https://github.com/astral-sh/uv). With a simple server-grade setup (Intel 8th Gen CPU), we can see [**uv**](https://github.com/astral-sh/uv) is 200x faster than pip:
3133

32-
| \# with cached packages, clean virtual environment $ time pip install vllm ... pip install vllm 59.09s user 3.82s system 83% cpu 1:15.68 total \# with cached packages, clean virtual environment $ time uv pip install vllm .... uv pip install vllm 0.17s user 0.57s system 193% cpu 0.383 total |
33-
| :---- |
34+
```sh
35+
# with cached packages, clean virtual environment
36+
$ time pip install vllm
37+
...
38+
pip install vllm 59.09s user 3.82s system 83% cpu 1:15.68 total
39+
40+
# with cached packages, clean virtual environment
41+
$ time uv pip install vllm
42+
...
43+
uv pip install vllm 0.17s user 0.57s system 193% cpu 0.383 total
44+
```
3445

3546
### Install the Latest vLLM from the Main Branch
3647

3748
To meet the community’s need for cutting-edge features and models, we provide nightly wheels for every commit on the main branch.
3849

3950
**Using pip**:
4051

41-
| pip install vllm \--pre \--extra-index-url https://wheels.vllm.ai/nightly |
42-
| :---- |
52+
```sh
53+
pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
54+
```
4355

44-
Adding \--pre ensures pip includes pre-released versions in its search.
56+
Adding `--pre` ensures pip includes pre-released versions in its search.
4557

4658
**Using uv**:
4759

48-
| uv pip install vllm \--extra-index-url https://wheels.vllm.ai/nightly |
49-
| :---- |
60+
```sh
61+
uv pip install vllm --extra-index-url https://wheels.vllm.ai/nightly
62+
```
5063

5164
## Development Made Simple
5265

@@ -56,10 +69,13 @@ We understand that an active, engaged developer community is the backbone of inn
5669

5770
For Python developers who need to tweak and test vLLM’s Python code, there’s no need to compile kernels. Our solution allows you to get started in just a few minutes:
5871

59-
| git clone https://github.com/vllm-project/vllm.git cd vllm VLLM\_USE\_PRECOMPILED=1 pip install \-e . |
60-
| :---- |
72+
```sh
73+
git clone https://github.com/vllm-project/vllm.git
74+
cd vllm
75+
VLLM_USE_PRECOMPILED=1 pip install -e .
76+
```
6177

62-
The VLLM\_USE\_PRECOMPILED=1 flag instructs the installer to use pre-compiled CUDA kernels instead of building them from source, significantly reducing installation time. This is perfect for developers focusing on Python-level features like API improvements, model support, or integration work.
78+
The `VLLM_USE_PRECOMPILED=1` flag instructs the installer to use pre-compiled CUDA kernels instead of building them from source, significantly reducing installation time. This is perfect for developers focusing on Python-level features like API improvements, model support, or integration work.
6379

6480
This lightweight process runs efficiently, even on a laptop. For more advanced usage, please check the [documentation](https://docs.vllm.ai/en/latest/getting_started/installation/gpu-cuda.html#python-only-build-without-compilation).
6581

@@ -69,23 +85,29 @@ For advanced contributors working with C++ code or CUDA kernels, we’ve optimiz
6985

7086
## Track Changes with Ease
7187

72-
The fast-evolving nature of LLM inference means interfaces and behaviors are still stabilizing. vLLM has been integrated into many workflows, including [OpenRLHF](https://github.com/OpenRLHF/OpenRLHF), [veRL](https://github.com/volcengine/verl), [open\_instruct](https://github.com/allenai/open-instruct), [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory), etc. We are working with them to stabilize interfaces and behaviors for LLM inference. To facilitate the process, we provide powerful tools for these powerful users to track changes across versions.
88+
The fast-evolving nature of LLM inference means interfaces and behaviors are still stabilizing. vLLM has been integrated into many workflows, including [OpenRLHF](https://github.com/OpenRLHF/OpenRLHF), [veRL](https://github.com/volcengine/verl), [open_instruct](https://github.com/allenai/open-instruct), [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory), etc. We are working with them to stabilize interfaces and behaviors for LLM inference. To facilitate the process, we provide powerful tools for these powerful users to track changes across versions.
7389

7490
### Installing a Specific Commit
7591

7692
To simplify tracking and testing, we provide wheels for every commit in the main branch. Users can easily install any specific commit, which can be particularly useful to bisect and track the changes.
7793

7894
We recommend using [**uv**](https://github.com/astral-sh/uv) to install a specific commit:
7995

80-
| \# use full commit hash from the main branch export VLLM\_COMMIT=72d9c316d3f6ede485146fe5aabd4e61dbc59069 uv pip install vllm \--extra-index-url https://wheels.vllm.ai/${VLLM\_COMMIT} |
81-
| :---- |
96+
```sh
97+
# use full commit hash from the main branch
98+
export VLLM_COMMIT=72d9c316d3f6ede485146fe5aabd4e61dbc59069
99+
uv pip install vllm --extra-index-url https://wheels.vllm.ai/${VLLM_COMMIT}
100+
```
82101

83-
In [**uv**](https://github.com/astral-sh/uv) , packages in `--extra-index-url` have [higher priority than the default index](https://docs.astral.sh/uv/pip/compatibility/#packages-that-exist-on-multiple-indexes), which makes it possible to install a developing version prior to the latest public release (at the time of writing, it is v0.6.6.post1).
102+
In [**uv**](https://github.com/astral-sh/uv), packages in `--extra-index-url` have [higher priority than the default index](https://docs.astral.sh/uv/pip/compatibility/#packages-that-exist-on-multiple-indexes), which makes it possible to install a developing version prior to the latest public release (at the time of writing, it is v0.6.6.post1).
84103

85-
In contrast, pip combines packages from \--extra-index-url and the default index, choosing only the latest version, which makes it difficult to install a developing version prior to the released version. Therefore, for pip users, it requires specifying a placeholder wheel name to install a specific commit:
104+
In contrast, pip combines packages from `--extra-index-url` and the default index, choosing only the latest version, which makes it difficult to install a developing version prior to the released version. Therefore, for pip users, it requires specifying a placeholder wheel name to install a specific commit:
86105

87-
| \# use full commit hash from the main branch export VLLM\_COMMIT=33f460b17a54acb3b6cc0b03f4a17876cff5eafd pip install https://wheels.vllm.ai/${VLLM\_COMMIT}/vllm-1.0.0.dev-cp38-abi3-manylinux1\_x86\_64.whl |
88-
| :---- |
106+
```sh
107+
# use full commit hash from the main branch
108+
export VLLM_COMMIT=33f460b17a54acb3b6cc0b03f4a17876cff5eafd
109+
pip install https://wheels.vllm.ai/${VLLM_COMMIT}/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl
110+
```
89111

90112
## Conclusion
91113

0 commit comments

Comments
 (0)