Skip to content

Commit 70ada93

Browse files
committed
add setup
Signed-off-by: yiliu30 <yi4.liu@intel.com>
1 parent a272292 commit 70ada93

File tree

1 file changed

+14
-0
lines changed
  • examples/pytorch/nlp/huggingface_models/language-modeling/quantization/auto_round/qwen

1 file changed

+14
-0
lines changed
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
pip install -r requirements.txt
2+
pip install setuptools --upgrade
3+
pip install packaging --upgrade
4+
pip install -U "huggingface_hub[cli]"
5+
# Intall vllm
6+
git clone -b ds-fp8kv --single-branch --quiet https://github.com/yiliu30/vllm-fork.git && cd vllm-fork
7+
VLLM_USE_PRECOMPILED=1 pip install --editable . -v
8+
cd ..
9+
# Install auto-round
10+
git clone -b ds-fp8kv --single-branch --quiet https://github.com/intel/auto-round.git && cd auto-round
11+
pip install -e . -v
12+
cd ..
13+
# Uninstall flash_attn to avoid conflicts
14+
pip uninstall flash_attn -y

0 commit comments

Comments
 (0)