|
1 | 1 | ## About
|
2 | 2 |
|
3 |
| -This repo is designed as a vLLM plugin for Intel GPU (also know as XPU in pytorch) which will provide custom kernels used in vLLM. |
| 3 | +This repo is designed as a vLLM plugin which provides custom kernels for Intel GPU (known as XPU in PyTorch). |
4 | 4 |
|
5 | 5 | ## Getting started
|
| 6 | +Currently we use PyTorch 2.8, oneapi 2025.1. |
6 | 7 |
|
7 |
| -current we use torch 2.8, oneapi is 2025.1. |
| 8 | +### How it works |
| 9 | +python3 setup.py build - will build a `_C.abi3.so` under build directory |
| 10 | +python3 setup.py install - will copy above .so to `vllm_xpu_kernels` folder |
| 11 | +python3 setup.py develop - will be local install if we use develop build or system/virtual env lib path if we use install. |
8 | 12 |
|
9 |
| -### how this project works |
10 |
| -Build system of this project will build a `_C.abi3.so` under build dir, and install step will copy this to `vllm_xpu_kernels` folder(will be local install if we use develop build or system/virtual env lib path if we use install). On vllm side, we will |
11 |
| -`import vllm_xpu_kernels._C` at start time which should register all custom ops so we can directly use. |
| 13 | +On vllm side, we will `import vllm_xpu_kernels._C` at start time which should register all custom ops so we can directly use. |
12 | 14 |
|
13 | 15 | ### prepare
|
14 | 16 |
|
15 | 17 | Install oneapi 2025.1 deep learning essential [dependency](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html).
|
16 | 18 |
|
17 |
| -create a new virtual env, install build dependency and torch dependency |
| 19 | +Create a new virtual env, install build dependency and torch dependency |
18 | 20 |
|
19 | 21 | ```
|
20 | 22 | pip install -r requirements.txt
|
21 | 23 | ```
|
22 | 24 |
|
23 | 25 | ### build & install
|
24 |
| -development install command (which will install to current dir): |
| 26 | +Build development installation to current directory: |
25 | 27 |
|
26 | 28 | ```
|
27 | 29 | VLLM_TARGET_DEVICE=xpu python3 setup.py develop
|
28 | 30 | ```
|
29 | 31 |
|
30 |
| -or |
31 |
| - |
32 |
| -install to system dir |
| 32 | +or installation to system directory: |
33 | 33 |
|
34 | 34 | ```
|
35 | 35 | VLLM_TARGET_DEVICE=xpu python3 setup.py install
|
36 | 36 | ```
|
37 | 37 |
|
38 |
| -or build wheel(generate on dist folder) |
| 38 | +or build wheel (generated .whl in dist folder) |
39 | 39 |
|
40 | 40 | ```
|
41 | 41 | VLLM_TARGET_DEVICE=xpu python3 setup.py bdist_wheel
|
42 | 42 | ```
|
43 | 43 |
|
44 | 44 | ### how to use in vLLM
|
45 |
| - |
46 |
| -please use this branch https://github.com/jikunshang/vllm/tree/xpu_kernel to install & test vllm. which already replace rms norm kernel from ipex to vllm-xpu-kernels. |
| 45 | +Please refer to temporary branch https://github.com/jikunshang/vllm/tree/xpu_kernel to install & test vllm which replaces `rms_norm` kernel from IPEX to vllm-xpu-kernels. |
0 commit comments