Skip to content

Commit a80bb77

Browse files
authored
Merge pull request #2537 from nikhil-arm/vllm_int4_fix
[fix]: fix env issues with vLLM int4 acceleration LP
2 parents d0db28e + 1b34e40 commit a80bb77

File tree

2 files changed

+5
-4
lines changed

2 files changed

+5
-4
lines changed

content/learning-paths/servers-and-cloud-computing/vllm-acceleration/1-overview-and-build.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,8 @@ Install the minimum system package used by vLLM on Arm:
4242

4343
```bash
4444
sudo apt-get update -y
45-
sudo apt-get install -y libnuma-dev
45+
sudo apt-get install -y build-essential cmake libnuma-dev
46+
sudo apt install python3.12-venv python3.12-dev
4647
```
4748

4849
Optional performance helper you can install now or later:
@@ -60,9 +61,9 @@ On aarch64, vLLM’s CPU backend automatically builds with Arm Compute Library v
6061
Create and activate a virtual environment:
6162

6263
```bash
63-
python3 -m venv vllm_env
64+
python3.12 -m venv vllm_env
6465
source vllm_env/bin/activate
65-
python -m pip install --upgrade pip
66+
python3 -m pip install --upgrade pip
6667
```
6768

6869
Clone vLLM and install build requirements:

content/learning-paths/servers-and-cloud-computing/vllm-acceleration/2-quantize-model.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -135,7 +135,7 @@ This script creates a Arm KleidiAI 4‑bit quantized copy of the vLLM model and
135135

136136
```bash
137137
# DeepSeek example
138-
python quantize_vllm_models.py deepseek-ai/DeepSeek-V2-Lite \
138+
python3 quantize_vllm_models.py deepseek-ai/DeepSeek-V2-Lite \
139139
--scheme channelwise --method mse
140140
```
141141

0 commit comments

Comments
 (0)