Skip to content

Commit a7432ad

Browse files
committed
fix docs
1 parent 49919b5 commit a7432ad

File tree

2 files changed

+2
-6
lines changed

2 files changed

+2
-6
lines changed

docs/source/Megatron-SWIFT/快速开始.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,6 @@ ms-swift引入了Megatron的并行技术来加速大模型的训练,包括数
1818
使用Megatron-SWIFT,除了安装swift依赖外,还需要安装以下内容:
1919

2020
```shell
21-
# 推荐torch版本:2.5 / 2.6
2221
pip install pybind11
2322

2423
# transformer_engine
@@ -30,8 +29,6 @@ pip install --no-build-isolation transformer_engine[pytorch]
3029
# apex
3130
git clone https://github.com/NVIDIA/apex
3231
cd apex
33-
# https://github.com/modelscope/ms-swift/issues/4176
34-
git checkout e13873debc4699d39c6861074b9a3b2a02327f92
3532
pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" ./
3633

3734
# megatron-core
@@ -50,6 +47,7 @@ export MEGATRON_LM_PATH='/xxx/Megatron-LM'
5047
# flash_attn
5148
# 选择合适的版本进行安装:https://github.com/Dao-AILab/flash-attention/releases/tag/v2.8.1
5249
# 注意:请勿安装高于transformer_engine限制的最高版本:https://github.com/NVIDIA/TransformerEngine/blob/release_v2.6/transformer_engine/pytorch/attention/dot_product_attention/utils.py#L109
50+
MAX_JOBS=8 pip install "flash-attn<2.8.2" --no-build-isolation
5351
```
5452

5553
或者你也可以使用镜像:(历史镜像查看[这里](../GetStarted/SWIFT安装.md#镜像)

docs/source_en/Megatron-SWIFT/Quick-start.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@ ms-swift incorporates Megatron's parallelization techniques to accelerate the tr
1717
To use Megatron-SWIFT, in addition to installing the `swift` dependencies, you also need to install the following:
1818

1919
```shell
20-
# Recommended PyTorch version: 2.5 / 2.6
2120
pip install pybind11
2221

2322
# transformer_engine
@@ -29,8 +28,6 @@ pip install --no-build-isolation transformer_engine[pytorch]
2928
# apex
3029
git clone https://github.com/NVIDIA/apex
3130
cd apex
32-
# https://github.com/modelscope/ms-swift/issues/4176
33-
git checkout e13873debc4699d39c6861074b9a3b2a02327f92
3431
pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" ./
3532

3633
# megatron-core
@@ -49,6 +46,7 @@ export MEGATRON_LM_PATH='/xxx/Megatron-LM'
4946
# flash_attn
5047
# Choose an appropriate version to install: https://github.com/Dao-AILab/flash-attention/releases/tag/v2.8.1
5148
# Note: Do not install a version higher than the maximum supported by transformer_engine: https://github.com/NVIDIA/TransformerEngine/blob/release_v2.6/transformer_engine/pytorch/attention/dot_product_attention/utils.py#L109
49+
MAX_JOBS=8 pip install "flash-attn<2.8.2" --no-build-isolation
5250
```
5351

5452
Alternatively, you can also use the image: (See historical images [here](../GetStarted/SWIFT-installation.md#mirror))

0 commit comments

Comments
 (0)