Skip to content

Commit 242086b

Browse files
move fsdp config path (#662)
1 parent cb45fad commit 242086b

File tree

5 files changed

+4
-9
lines changed

5 files changed

+4
-9
lines changed

docs/source/LLM/命令行参数.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,7 @@
108108
### FSDP参数
109109

110110
- `--fsdp`: 默认值`''`, fsdp类型, 详情可以查看该参数[原始文档](https://huggingface.co/docs/transformers/v4.39.3/en/main_classes/trainer#transformers.TrainingArguments.fsdp).
111-
- `--fsdp_config`: 默认值`None`, fsdp配置文件的路径, 支持传入`fsdp_offload`, 该文件为SWIFT提供的默认配置, 具体可以查看[这里](https://github.com/modelscope/swift/tree/main/swift/llm/fsdp_config/fsdp_offload.json).
111+
- `--fsdp_config`: 默认值`None`, fsdp配置文件的路径.
112112

113113
### LoRA+微调参数
114114

docs/source_en/LLM/Command-line-parameters.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -107,9 +107,9 @@
107107

108108
### FSDP Parameters
109109

110-
- `--fsdp`: Default value`''`, the FSDP type, please check[this documentation](https://huggingface.co/docs/transformers/v4.39.3/en/main_classes/trainer#transformers.TrainingArguments.fsdp) for details.
110+
- `--fsdp`: Default value `''`, the FSDP type, please check [this documentation](https://huggingface.co/docs/transformers/v4.39.3/en/main_classes/trainer#transformers.TrainingArguments.fsdp) for details.
111111

112-
- `--fsdp_config`: Default value`None`, the FSDP config file path, `fsdp_offload` is a special value, check [here](https://github.com/modelscope/swift/tree/main/swift/llm/fsdp_config/fsdp_offload.json) for details.
112+
- `--fsdp_config`: Default value `None`, the FSDP config file path.
113113

114114
### LoRA+ Fine-tuning Parameters
115115

examples/pytorch/llm/scripts/llama2_70b_chat/qlora_fsdp/sft.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ nproc_per_node=2
44

55
PYTHONPATH=../../.. \
66
CUDA_VISIBLE_DEVICES=0,1 \
7-
accelerate launch --config_file "../../../swift/llm/fsdp_config/fsdp_offload.json" \
7+
accelerate launch --config_file "./scripts/llama2_70b_chat/qlora_fsdp/fsdp_offload.json" \
88
llm_sft.py \
99
--model_type llama2-70b-chat \
1010
--model_revision master \

swift/llm/utils/argument.py

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -290,11 +290,6 @@ def __post_init__(self) -> None:
290290
self.deepspeed = os.path.abspath(
291291
os.path.join(ds_config_folder, 'zero3.json'))
292292

293-
fsdp_config_folder = os.path.join(__file__, '..', '..', 'fsdp_config')
294-
if self.fsdp_config == 'fsdp_offload':
295-
self.fsdp_config = os.path.abspath(
296-
os.path.join(fsdp_config_folder, 'fsdp_offload.json'))
297-
298293
handle_path(self)
299294
set_model_type(self)
300295
if isinstance(self.dataset, str):

0 commit comments

Comments
 (0)