@@ -119,14 +119,41 @@ conda activate worldplay
119119pip install -r requirements.txt
120120```
121121
122- ### 2. Install Flash Attention (Optional but Recommended)
123- Install Flash Attention for faster inference and reduced GPU memory consumption:
124- ``` bash
125- pip install flash-attn --no-build-isolation
126- ```
127- Detailed instructions: [ Flash Attention] ( https://github.com/Dao-AILab/flash-attention )
128-
129- ### 3. Download All Required Models
122+ ### 2. Install Attention Libraries (Optional but Recommended)
123+ * Flash Attention:
124+ Install Flash Attention for faster inference and reduced GPU memory consumption:
125+ ``` bash
126+ pip install flash-attn --no-build-isolation
127+ ```
128+ Detailed instructions: [ Flash Attention] ( https://github.com/Dao-AILab/flash-attention )
129+
130+
131+ * SageAttention:
132+ To enable SageAttention for faster inference, you need to install it by the following command:
133+ ``` bash
134+ git clone https://github.com/cooper1637/SageAttention.git
135+ cd SageAttention
136+ export EXT_PARALLEL=4 NVCC_APPEND_FLAGS=" --threads 8" MAX_JOBS=32 # Optional
137+ python3 setup.py install
138+ ```
139+
140+ ### 3. Install AngelSlim and DeepGEMM
141+ * AngelSlim:
142+ Install AngelSlim to quantize transformer.
143+ ``` bash
144+ pip install angelslim==0.2.2
145+ ```
146+
147+ * DeepGEMM:
148+ To enable fp8 gemm for transformer, you need to install it by the following command:
149+ ``` bash
150+ git clone --recursive git@github.com:deepseek-ai/DeepGEMM.git
151+ cd DeepGEMM
152+ ./develop.sh
153+ ./install.sh
154+ ```
155+
156+ ### 4. Download All Required Models
130157
131158We provide a download script that automatically downloads all required models:
132159
@@ -315,7 +342,6 @@ https://github.com/user-attachments/assets/531bf0ad-1fca-4d76-bb65-84701368926d
315342https://github.com/user-attachments/assets/f165f409-5a74-4e19-a32c-fc98d92259e1
316343
317344## 📝 TODO
318- - [ ] Acceleration & Quantization
319345- [ ] Open-source training code
320346
321347## 📚 Citation
0 commit comments