We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
1 parent a272292 commit 70ada93Copy full SHA for 70ada93
examples/pytorch/nlp/huggingface_models/language-modeling/quantization/auto_round/qwen/setup.sh
@@ -0,0 +1,14 @@
1
+pip install -r requirements.txt
2
+pip install setuptools --upgrade
3
+pip install packaging --upgrade
4
+pip install -U "huggingface_hub[cli]"
5
+# Intall vllm
6
+git clone -b ds-fp8kv --single-branch --quiet https://github.com/yiliu30/vllm-fork.git && cd vllm-fork
7
+VLLM_USE_PRECOMPILED=1 pip install --editable . -v
8
+cd ..
9
+# Install auto-round
10
+git clone -b ds-fp8kv --single-branch --quiet https://github.com/intel/auto-round.git && cd auto-round
11
+pip install -e . -v
12
13
+# Uninstall flash_attn to avoid conflicts
14
+pip uninstall flash_attn -y
0 commit comments