Skip to content

Commit cd578d6

Browse files
ILikeIneineILikeIneine
committed
[docs] update v0.13.0 docs
Signed-off-by: Hank <hcc.mayday@gmail.com>
1 parent 23dd592 commit cd578d6

File tree

5 files changed

+200
-72
lines changed

5 files changed

+200
-72
lines changed

‎README.md‎

Lines changed: 18 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,9 @@ vLLM MetaX Plugin
1717
---
1818

1919
*Latest News* 🔥
20-
20+
- [2026/2] Released vllm-metax **v0.13.0** 🧨 — aligned with vLLM *v0.13.0*, brings you the latest features and model in v0.13.0!
21+
- [2026/1] Released vllm-metax **v0.12.0** 😎 — aligned with vLLM *v0.12.0*, supported more models and improved performance.
22+
- [2026/1] Released vllm-metax **v0.11.2** 👻 — aligned with vLLM *v0.11.2*, supported more models and improved performance.
2123
- [2025/11] Released vllm-metax **v0.10.2** 🎉 — aligned with vLLM *v0.10.2*, improved model performance, and fixed key decoding bugs.
2224
- [2025/11] We hosted [vLLM Beijing Meetup](https://mp.weixin.qq.com/s/xSrYXjNgr1HbCP4ExYNG1w) focusing on distributed inference and diverse accelerator support with vLLM! Please find the meetup slides [here](https://drive.google.com/drive/folders/1nQJ8ZkLSjKxvu36sSHaceVXtttbLvvu-?usp=drive_link).
2325
- [2025/08] We hosted [vLLM Shanghai Meetup](https://mp.weixin.qq.com/s/pDmAXHcN7Iqc8sUKgJgGtg) focusing on building, developing, and integrating with vLLM! Please find the meetup slides [here](https://drive.google.com/drive/folders/1OvLx39wnCGy_WKq8SiVKf7YcxxYI3WCH).
@@ -46,29 +48,33 @@ Which ensured the hardware features and functionality support on integration of
4648

4749
## Getting Started
4850

49-
vLLM MetaX currently only support starting on docker images release by [MetaX develop community](https://developer.metax-tech.com/softnova/docker) which is out of box. (DockerFile for other OS is undertesting)
51+
vLLM MetaX currently only support starting on docker images release by [MetaX develop community](https://developer.metax-tech.com/softnova/docker?chip_name=%E6%9B%A6%E4%BA%91C500%E7%B3%BB%E5%88%97&package_kind=AI&dimension=docker&deliver_type=%E5%88%86%E5%B1%82%E5%8C%85&ai_frame=vllm-metax) which is out of box. (DockerFile for other OS is undertesting)
5052

51-
If you want to develop, debug or test the newest feature on vllm-metax, you may need to build from scratch and follow this [*source build tutorial*](https://vllm-metax.readthedocs.io/en/latest/getting_started/installation/maca.html).
53+
If you want to develop, debug or test the newest feature on vllm-metax, you may need to build from scratch and follow this [*source build tutorial*](https://vllm-metax.readthedocs.io/en/v0.13.0/getting_started/installation/maca.html).
5254

5355
## Branch
5456

55-
vllm-metax has master branch and dev branch.
57+
vllm-metax has three kind of branches.
5658

5759
- **master**: main branch，catching up with main branch of vLLM upstream.
58-
- **vX.Y.Z-dev**: development branch, created with part of new releases of vLLM. For example, `v0.10.2-dev` is the dev branch for vLLM `v0.10.2` version.
60+
- **releases/vX.Y.Z**: release branch, created when a new version of vLLM is released. For example, `releases/v0.13.0` is the release branch for vLLM `v0.13.0` version. (Same tag name)
61+
- **vX.Y.Z-dev**: development branch, created with part of new releases of vLLM. For example, `v0.14.0-dev` is the dev branch for vLLM `v0.14.0` version.
5962

6063
Below is maintained branches:
6164

6265
| Branch | Status | Note |
6366
|-------------|--------------|--------------------------------------|
64-
| master | Maintained | trying to support vllm main, no gurantee on functionality |
65-
| v0.11.1rc6-dev| Maintained | under testing |
66-
| v0.11.0-dev | Maintained | under testing |
67-
| releases/v0.10.2 | Maintained | Released |
67+
| master | N/A | trying to support vllm main, no gurantee on functionality |
68+
| v0.15.0-dev | N/A | under testing |
69+
| v0.14.0-dev | N/A | under testing |
70+
| releases/v0.13.0 | Released | related to vllm release v0.13.0 |
71+
| releases/v0.12.0 | Released | related to vllm release v0.12.0 |
72+
| releases/v0.11.2 | Released | related to vllm release v0.11.2 |
73+
| releases/v0.10.2 | Released | related to vllm release v0.10.2 |
6874

69-
Please check [here](https://vllm-metax.readthedocs.io/en/latest/getting_started/quickstart.html) for more details .
7075

71-
## License
76+
Please check [here](https://vllm-metax.readthedocs.io/en/v0.13.0/getting_started/quickstart.html) for v0.13.0 details.
7277

73-
Apache License 2.0, as found in the [LICENSE](./LICENSE) file.
78+
## License
7479

80+
Apache License 2.0, as found in the [LICENSE](./LICENSE) file.

‎docs/getting_started/installation/maca.md‎

Lines changed: 42 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -1,83 +1,80 @@
11
# Installation
22

3+
!!! warning "Breaking Change Notice"
4+
After v0.11.2, vLLM-MetaX moved its `_C` and `_moe_C` kernel into a separate package named `mcoplib`.
5+
6+
mcoplib is open-sourced at [MetaX-mcoplib](https://github.com/MetaX-MACA/mcoplib) and would maintain its own release cycle. vllm-metax's release rely on its corresponding version of mcoplib. Check it at the [Release Page](../quickstart.md#releases).
7+
8+
Though the *csrc* folder is still kept in this repo for development convenience, and there is no guarantee that the code is always in sync with mcoplib. Not only the performance but also the correctness may differ from mcoplib.
9+
10+
To build and use the vllm-metax csrc , you need to set:
11+
12+
```bash
13+
export USE_PRECOMPILED_KERNEL=0
14+
```
15+
16+
in both *build* and *runtime* environment variables.
17+
18+
**Please always use mcoplib for production usage.**
19+
320
## Requirements
421

522
- OS: Linux
623
- Python: 3.10 -- 3.12
724

8-
## Set up using pip (without UV)
9-
10-
### Build wheel from source
11-
12-
!!! note
13-
If using pip, all the build and installation steps are based on *corresponding docker images*. You can find them on [quick start](../quickstart.md).
14-
We need to add `-no-build-isolation` flag (or an equivalent one) during package building, since all the requirements are already pre-installed in released docker image.
1525

16-
#### Setup environment variables
26+
## Build from source
1727

28+
### Prepare environment
1829
```bash
1930
# setup MACA path
2031
export MACA_PATH="/opt/maca"
2132

2233
# cu-bridge
2334
export CUCC_PATH="${MACA_PATH}/tools/cu-bridge"
24-
export CUDA_PATH=/root/cu-bridge/CUDA_DIR
35+
export CUDA_PATH="${HOME}/cu-bridge/CUDA_DIR"
2536
export CUCC_CMAKE_ENTRY=2
2637

2738
# update PATH
2839
export PATH=${MACA_PATH}/mxgpu_llvm/bin:${MACA_PATH}/bin:${CUCC_PATH}/tools:${CUCC_PATH}/bin:${PATH}
2940
export LD_LIBRARY_PATH=${MACA_PATH}/lib:${MACA_PATH}/ompi/lib:${MACA_PATH}/mxgpu_llvm/lib:${LD_LIBRARY_PATH}
30-
31-
export VLLM_INSTALL_PUNICA_KERNELS=1
3241
```
3342

34-
#### Build vllm
43+
=== "PIP"
44+
--8<-- "docs/getting_started/installation/pip.inc.md:prepare-env"
45+
=== "UV"
46+
--8<-- "docs/getting_started/installation/uv.inc.md:prepare-env"
47+
48+
### Build vllm
3549

3650
Clone vllm project:
3751

3852
```bash
39-
git clone --depth 1 --branch main https://github.com/vllm-project/vllm
53+
git clone --depth 1 --branch releases/v0.13.0 https://github.com/vllm-project/vllm
4054
cd vllm
4155
```
4256

4357
Build with *empty device*:
4458

45-
```bash
46-
python use_existing_torch.py
47-
pip install -r requirements/build.txt
48-
VLLM_TARGET_DEVICE=empty pip install -v . --no-build-isolation
49-
```
59+
=== "PIP"
60+
--8<-- "docs/getting_started/installation/pip.inc.md:build-vllm"
61+
=== "UV"
62+
--8<-- "docs/getting_started/installation/uv.inc.md:build-vllm"
5063

51-
#### Build plugin
64+
### Build plugin
5265

53-
Install the build requirments first:
66+
Clone vllm-metax project:
5467

55-
```bash
56-
python use_existing_metax.py
57-
pip install -r requirements/build.txt
58-
```
59-
60-
Build and install vLLM:
61-
62-
```bash
63-
pip install . -v --no-build-isolation
64-
```
65-
66-
If you want to develop vLLM, install it in editable mode instead.
67-
68-
```bash
69-
pip install . -e -v --no-build-isolation
68+
```bash
69+
git clone --branch releases/v0.13.0 https://github.com/MetaX-MACA/vLLM-metax
70+
cd vLLM-metax
7071
```
7172

72-
Optionally, build a portable wheel which you can then install elsewhere:
73-
74-
```bash
75-
python -m build -w -n
76-
pip install dist/*.whl
77-
```
78-
79-
## Set up using UV (experimental)
73+
Build the plugin:
8074

81-
Todo
75+
=== "PIP"
76+
--8<-- "docs/getting_started/installation/pip.inc.md:build-vllm-metax"
77+
=== "UV"
78+
--8<-- "docs/getting_started/installation/uv.inc.md:build-vllm-metax"
8279

83-
## Extra information
80+
## Extra information

‎docs/getting_started/installation/pip.inc.md‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# --8<-- [start:prepare-env]
2+
!!! note
3+
If using pip, all the build and installation steps are ***based on corresponding docker images***. You can find them on [QuickStart page](../quickstart.md).
4+
We need to add `--no-build-isolation` flag during the whole package building since we need all the requirements that were pre-installed in released docker image.
5+
# --8<-- [end:prepare-env]
6+
7+
# --8<-- [start:build-vllm-metax]
8+
!!! note
9+
10+
```bash
11+
python use_existing_metax.py
12+
pip install -r requirements/build.txt
13+
pip install . --no-build-isolation
14+
```
15+
16+
??? console "Additional installation options"
17+
If you want to develop vllm-metax, install it in **editable mode** instead.
18+
19+
```bash
20+
pip install -v -e . --no-build-isolation
21+
```
22+
23+
Optionally, build a portable wheel which you can then install elsewhere.
24+
25+
```bash
26+
python -m build -w -n
27+
pip install dist/*.whl
28+
```
29+
# --8<-- [end:build-vllm-metax]
30+
31+
# --8<-- [start:build-vllm]
32+
!!! note "To build vllm-metax using an existing PyTorch installation"
33+
34+
```bash
35+
python use_existing_pytorch.py
36+
pip install -r requirements/build.txt
37+
VLLM_TARGET_DEVICE=empty pip install . --no-build-isolation
38+
```
39+
# --8<-- [end:build-vllm]

‎docs/getting_started/installation/uv.inc.md‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# --8<-- [start:prepare-env]
2+
!!! note
3+
4+
UV **does not rely** on any pre-installed packages in the docker, and would install all the dependencies in a virtual environment from scratch.
5+
6+
??? console "UV installation guide"
7+
We'd recommend install uv with pip (this is not forcibly required):
8+
9+
```bash
10+
pip install uv
11+
```
12+
13+
Then create the virtual environment with python 3.10 or above:
14+
15+
```bash
16+
uv venv /opt/venv --python python3.10
17+
```
18+
19+
And activate the virtual environment:
20+
21+
```bash
22+
source /opt/venv/bin/activate
23+
```
24+
25+
You need to manually set Metax PyPi repo to download maca-related dependencies.
26+
27+
```
28+
export UV_EXTRA_INDEX_URL=https://repos.metax-tech.com/r/maca-pypi/simple
29+
export UV_INDEX_STRATEGY=unsafe-best-match
30+
```
31+
32+
??? console "Optional: Change PyPi default mirror"
33+
You could set Aliyun PyPi mirror as default to speed up package downloading:
34+
35+
```bash
36+
export UV_INDEX_URL=https://mirrors.aliyun.com/pypi/simple
37+
```
38+
# --8<-- [end:prepare-env]
39+
40+
# --8<-- [start:build-vllm-metax]
41+
!!! note
42+
43+
```bash
44+
uv pip install -r requirements/build.txt
45+
uv pip install .
46+
```
47+
48+
??? console "Additional installation options"
49+
If you want to develop vLLM, install it in editable mode instead.
50+
51+
```bash
52+
uv pip install -v -e .
53+
```
54+
55+
Optionally, build a portable wheel which you can then install elsewhere.
56+
57+
```bash
58+
uv build --wheel
59+
```
60+
# --8<-- [end:build-vllm-metax]
61+
62+
63+
64+
# --8<-- [start:build-vllm]
65+
!!! note "To build vLLM using local uv environment"
66+
67+
```bash
68+
uv pip install -r requirements/build.txt
69+
VLLM_TARGET_DEVICE=empty uv pip install . --no-build-isolation
70+
```
71+
72+
??? note "About isolation"
73+
`--no-build-isolation` is optional. we add this option for speeding up installation.
74+
uv would still trying to download cuda-related packages during build even if you set
75+
`VLLM_TARGET_DEVICE=empty`, which may take a long time.
76+
77+
# --8<-- [end:build-vllm]

‎docs/getting_started/quickstart.md‎

Lines changed: 24 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -2,26 +2,35 @@
22

33
Currently the recommanded way to start ***vLLM-MetaX*** is via *docker*.
44

5-
You could get the docker image at [MetaX develop community](https://developer.metax-tech.com/softnova/docker).
5+
You could get the docker image at [MetaX develop community](https://developer.metax-tech.com/softnova/docker?chip_name=%E6%9B%A6%E4%BA%91C500%E7%B3%BB%E5%88%97&package_kind=AI&dimension=docker&deliver_type=%E5%88%86%E5%B1%82%E5%8C%85&ai_frame=vllm-metax&arch=amd64&system=ubuntu).
66

7-
*Belows is version mapping to released plugin and maca*:
7+
!!! note
8+
After v0.11.2, vllm-metax moved its `_C` and `_moe_C` kernel into a separate package named `mcoplib`.
9+
10+
**mcoplib** is open-sourced at [MetaX-mcoplib](https://github.com/MetaX-MACA/mcoplib) and would maintain its own release cycle. Please always install the corresponding version of mcoplib when using vLLM-MetaX.
811

9-
| plugin version | maca version | docker distribution tag |
10-
|:--------------:|:------------:|:-----------------------:|
11-
|v0.8.5 |maca2.33.1.13 | vllm:maca.ai2.33.1.13-torch2.6-py310-ubuntu22.04-amd64 |
12-
|v0.9.1 |maca3.0.0.5 | vllm:maca.ai3.0.0.5-torch2.6-py310-ubuntu22.04-amd64 |
13-
|v0.10.1.1 (dev) |maca3.1.0.7 | not released |
14-
|v0.10.2 |maca3.2.0.7 | not released |
15-
|v0.11.0 |maca3.2.x.x | not released |
16-
|master |maca3.2.x.x. | not released |
12+
Though the *csrc* folder is still kept in this repo for development convenience, and there is no guarantee that the code is always in sync with mcoplib. Not only the performance but also the correctness may differ from mcoplib.
1713

18-
> Note: All the vllm tests are based on the related maca version. Using incorresponding version of maca for vllm may cause unexpected bugs or errors. This is not garanteed.
14+
If you want build the latest vllm-metax, please refer to [installation](./installation/maca.md) to build from source.
1915

20-
vLLM-MetaX is out of box via these docker images.
16+
**Please always use mcoplib for production usage.**
2117

22-
## Offline Batched Inference
18+
## Releases
2319

24-
## OpenAI-Compatible Server
20+
*Belows is version mapping to released plugin and mcoplib with maca*:
2521

26-
## On Attention Backends
22+
| plugin version | maca version | mcoplib version | docker distribution tag |
23+
|:--------------:|:------------:|:-----------------------:|:-----------------------:|
24+
|v0.8.5 |maca2.33.1.13 | N/A | vllm:maca.ai2.33.1.13-torch2.6-py310-ubuntu22.04-amd64 |
25+
|v0.9.1 |maca3.0.0.5 | N/A | vllm:maca.ai3.0.0.5-torch2.6-py310-ubuntu22.04-amd64 |
26+
|v0.10.2 |maca3.2.1.7 | N/A | vllm-metax:0.10.2-maca.ai3.2.1.7-torch2.6-py310-ubuntu22.04-amd64 |
27+
|v0.11.0 |maca3.3.0.15 | 0.1.1 | vllm-metax:0.11.0-maca.ai3.3.0.11-torch2.6-py312-ubuntu22.04-amd64 |
28+
|v0.11.2 |maca3.3.0.15 | 0.2.0 | vllm-metax:0.11.2-maca.ai3.3.0.103-torch2.8-py312-ubuntu22.04-amd64 |
29+
|v0.12.0 |maca3.3.0.15 | 0.3.0 | vllm-metax:0.12.0-maca.ai3.3.0.204-torch2.8-py312-ubuntu22.04-amd64 |
30+
|v0.13.0 |maca3.3.0.15 | 0.3.1 | vllm-metax:0.13.0-maca.ai3.3.0.303-torch2.8-py312-ubuntu22.04-amd64 |
31+
|master |maca3.3.0.15 | >=0.3.0 | not released |
2732

33+
!!! warning "Usage Warning"
34+
**vLLM-MetaX is out of box via these docker images.**
35+
36+
All the vllm tests are based on the related maca version. Using incorresponding version of maca for vllm may cause unexpected bugs or errors. This is not garanteed.

0 commit comments

Comments
 (0)