Skip to content

Commit ff2210e

Browse files
authored
Update README.md with more accurate descriptions (#52)
1 parent fe3db17 commit ff2210e

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -160,8 +160,8 @@ python sample.py \
160160
![fastseq_overview](./figure/fastseq_overview.png)
161161

162162
In the realm of visual generation models, such as DiT, sequence parallelism is indispensable for effective long-sequence training and low-latency inference. Two key features can summarize the distinctive nature of these tasks:
163-
* The model parameter is small, but the sequence can be very long, making communication a bottleneck.
164-
* As the model size is generally small, it only needs sequence parallelism within a node.
163+
* The model parameter is smaller compared with LLMs, but the sequence can be very long, making communication a bottleneck.
164+
* As the model size is relatively small, it only needs sequence parallelism within a node.
165165

166166
However, existing methods like DeepSpeed-Ulysses and Megatron-LM Sequence Parallelism face limitations when applied to such tasks. They either introduce excessive sequence communication or lack efficiency in handling small-scale sequence parallelism.
167167

@@ -182,7 +182,7 @@ Our loss also aligns with the results listed in the paper:
182182

183183
![Loss](./figure/dit_loss.png)
184184

185-
To reproduce our results, you need to change the dataset to ImageNet in `train_img.py` and execute the following command:
185+
To reproduce our results, you need to change the dataset in `train_img.py` and execute the following command:
186186

187187
```
188188
torchrun --standalone --nproc_per_node=8 train.py \

0 commit comments

Comments
 (0)