Skip to content

Commit f239fb8

Browse files
committed
fix args.json (#4566)
1 parent 4878744 commit f239fb8

File tree

3 files changed

+5
-3
lines changed

3 files changed

+5
-3
lines changed

docs/source/Instruction/命令行参数.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -352,7 +352,7 @@ Vera使用`target_modules`, `target_regex`, `modules_to_save`三个参数.
352352
- check_model: 检查本地模型文件有损坏或修改并给出提示,默认为True。如果是断网环境,请设置为False。
353353
- 🔥create_checkpoint_symlink: 额外创建checkpoint软链接,方便书写自动化训练脚本。best_model和last_model的软链接路径分别为f'{output_dir}/best'和f'{output_dir}/last'。
354354
- loss_type: loss类型。默认为None,使用模型自带损失函数。
355-
- channels : 数据集包含的channel集合。默认为None。结合`--loss_type channel_loss`使用,可参考[这里](https://github.com/modelscope/ms-swift/blob/main/examples/train/plugins/channel_loss.sh)
355+
- channels: 数据集包含的channel集合。默认为None。结合`--loss_type channel_loss`使用,可参考[这里](https://github.com/modelscope/ms-swift/blob/main/examples/train/plugins/channel_loss.sh)
356356
- 🔥packing: 是否使用序列packing提升计算效率,默认为False。当前支持`swift pt/sft`
357357
- 注意:使用packing请结合`--attn_impl flash_attn`使用且"transformers>=4.44",具体查看[该PR](https://github.com/huggingface/transformers/pull/31629)
358358
- 支持的多模态模型参考:https://github.com/modelscope/ms-swift/blob/main/examples/train/packing/qwen2_5_vl.sh

docs/source_en/Instruction/Command-line-parameters.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -361,7 +361,7 @@ Training arguments include the [base arguments](#base-arguments), [Seq2SeqTraine
361361
- check_model: Check local model files for corruption or modification and give a prompt, default is True. If in an offline environment, please set to False.
362362
- 🔥create_checkpoint_symlink: Creates additional checkpoint symlinks to facilitate writing automated training scripts. The symlink paths for `best_model` and `last_model` are `f'{output_dir}/best'` and `f'{output_dir}/last'` respectively.
363363
- loss_type: Type of loss. Defaults to None, which uses the model's built-in loss function.
364-
- channelsSet of channels included in the dataset. Defaults to None. Used in conjunction with `--loss_type channel_loss`. Refer to [this example](https://github.com/modelscope/ms-swift/blob/main/examples/train/plugins/channel_loss.sh) for more details.
364+
- channels: Set of channels included in the dataset. Defaults to None. Used in conjunction with `--loss_type channel_loss`. Refer to [this example](https://github.com/modelscope/ms-swift/blob/main/examples/train/plugins/channel_loss.sh) for more details.
365365
- 🔥packing: Whether to use sequence packing to improve computational efficiency. The default value is False. Currently supports `swift pt/sft`.
366366
- Note: When using packing, please combine it with `--attn_impl flash_attn` and ensure "transformers>=4.44". For details, see [this PR](https://github.com/huggingface/transformers/pull/31629).
367367
- Supported multimodal models reference: https://github.com/modelscope/ms-swift/blob/main/examples/train/packing/qwen2_5_vl.sh

swift/llm/utils.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -252,9 +252,11 @@ def save_checkpoint(model: Optional[PreTrainedModel],
252252
if model and model.model_dir and model.model_dir not in model_dirs:
253253
model_dirs.append(model.model_dir)
254254
for src_file in (additional_saved_files or []) + ['preprocessor_config.json', 'args.json']:
255+
tgt_path = os.path.join(output_dir, src_file)
256+
if os.path.exists(tgt_path) and src_file == 'args.json':
257+
continue
255258
for model_dir in model_dirs:
256259
src_path: str = os.path.join(model_dir, src_file)
257-
tgt_path = os.path.join(output_dir, src_file)
258260
if os.path.isfile(src_path):
259261
shutil.copy(src_path, tgt_path)
260262
break

0 commit comments

Comments
 (0)