update readme (#386)

Jintao-Huang · web-flow · commit cc1a305e920a · 2024-02-06T01:03:04.000+08:00
diff --git a/README.md b/README.md
@@ -238,8 +238,8 @@ app_ui_main(infer_args)
     - [internlm2-math](https://github.com/InternLM/InternLM-Math) series: internlm2-math-7b, internlm2-math-7b-chat, internlm2-math-20b, internlm2-math-20b-chat.
 - Supported Datasets: [[Detailed Info]](https://github.com/modelscope/swift/blob/main/docs/source/LLM/%E6%94%AF%E6%8C%81%E7%9A%84%E6%A8%A1%E5%9E%8B%E5%92%8C%E6%95%B0%E6%8D%AE%E9%9B%86.md#%E6%95%B0%E6%8D%AE%E9%9B%86)
   - NLP:
-    - General: 🔥ms-bench, 🔥alpaca-en(gpt4), 🔥alpaca-zh(gpt4), multi-alpaca-all, instinwild-en, instinwild-zh, cot-en, cot-zh, firefly-all-zh, instruct-en, gpt4all-en, sharegpt-en, sharegpt-zh, tutu-v2-sft-mixture, wikipedia-zh, open-orca, open-orca-gpt4, sharegpt-gpt4.
-    - Agent: 🔥ms-agent, damo-agent-zh, 🔥agent-instruct-all-en.
+    - General: 🔥ms-bench, 🔥alpaca-en(gpt4), 🔥alpaca-zh(gpt4), multi-alpaca-all, instinwild-en, instinwild-zh, cot-en, cot-zh, firefly-all-zh, instruct-en, gpt4all-en, sharegpt-en, sharegpt-zh, 🔥tulu-v2-sft-mixture, wikipedia-zh, open-orca, open-orca-gpt4, sharegpt-gpt4.
+    - Agent: 🔥ms-agent, damo-mini-agent-zh, damo-agent-zh, agent-instruct-all-en.
     - RLHF: 🔥hh-rlhf, stack-exchange-paired.
     - Coding: code-alpaca-en, 🔥leetcode-python-en, 🔥codefuse-python-en, 🔥codefuse-evol-instruction-zh.
     - Medical: medical-en, medical-zh, medical-mini-zh.
diff --git a/README_CN.md b/README_CN.md
@@ -238,8 +238,8 @@ app_ui_main(infer_args)
     - [internlm2-math](https://github.com/InternLM/InternLM-Math) 系列: internlm2-math-7b, internlm2-math-7b-chat, internlm2-math-20b, internlm2-math-20b-chat.
 - 支持的数据集: [[详细信息]](https://github.com/modelscope/swift/blob/main/docs/source/LLM/%E6%94%AF%E6%8C%81%E7%9A%84%E6%A8%A1%E5%9E%8B%E5%92%8C%E6%95%B0%E6%8D%AE%E9%9B%86.md#%E6%95%B0%E6%8D%AE%E9%9B%86)
   - NLP:
-    - 通用: 🔥ms-bench, 🔥alpaca-en(gpt4), 🔥alpaca-zh(gpt4), multi-alpaca-all, instinwild-en, instinwild-zh, cot-en, cot-zh, firefly-all-zh, instruct-en, gpt4all-en, sharegpt-en, sharegpt-zh, tutu-v2-sft-mixture, wikipedia-zh, open-orca, open-orca-gpt4, sharegpt-gpt4.
-    - Agent: 🔥ms-agent, damo-agent-zh, 🔥agent-instruct-all-en.
+    - 通用: 🔥ms-bench, 🔥alpaca-en(gpt4), 🔥alpaca-zh(gpt4), multi-alpaca-all, instinwild-en, instinwild-zh, cot-en, cot-zh, firefly-all-zh, instruct-en, gpt4all-en, sharegpt-en, sharegpt-zh, 🔥tulu-v2-sft-mixture, wikipedia-zh, open-orca, open-orca-gpt4, sharegpt-gpt4.
+    - Agent: 🔥ms-agent, damo-mini-agent-zh, damo-agent-zh, agent-instruct-all-en.
     - RLHF: 🔥hh-rlhf, stack-exchange-paired.
     - 代码: code-alpaca-en, 🔥leetcode-python-en, 🔥codefuse-python-en, 🔥codefuse-evol-instruction-zh.
     - 医疗: medical-en, medical-zh, medical-mini-zh.
diff --git a/docs/source/LLM/命令行参数.md b/docs/source/LLM/命令行参数.md
@@ -52,7 +52,7 @@
 - `--lora_dtype`: 默认为`'fp32'`, 指定lora模块的dtype类型. 如果是`AUTO`则跟随原始模块的dtype类型. 你可以选择的值: 'fp16', 'bf16', 'fp32', 'AUTO'.
 - `--neftune_noise_alpha`: `NEFTune`添加的噪声系数, 可以提升模型在指令微调中的性能, 默认为`None`. 通常可以设置为5, 10, 15. 你可以查看[相关论文](https://arxiv.org/abs/2310.05914).
 - `--gradient_checkpointing`: 是否开启gradient checkpointing, 默认为`True`. 该参数可以用于节约显存, 虽然这会略微降低训练速度. 该参数在max_length较大, batch_size较大时作用显著.
-- `--deepspeed_config_path`: 用于指定deepspeed的配置文件的路径, 默认为`None`, 即不开启deepspeed. deepspeed可以节约显存. 我们书写了默认的[ZeRO-2配置文件](https://github.com/modelscope/swift/blob/main/examples/pytorch/llm/ds_config/zero2.json), [ZeRO-3配置文件](https://github.com/modelscope/swift/blob/main/examples/pytorch/llm/ds_config/zero3.json). 你只需要指定'default-zero2', 就会使用默认zero2配置文件; 指定'default-zero3', 就会使用默认的zero3配置文件.
+- `--deepspeed`: 用于指定deepspeed的配置文件的路径或者直接传入json格式的配置信息, 默认为`None`, 即不开启deepspeed. deepspeed可以节约显存. 我们书写了默认的[ZeRO-2配置文件](https://github.com/modelscope/swift/blob/main/examples/pytorch/llm/ds_config/zero2.json), [ZeRO-3配置文件](https://github.com/modelscope/swift/blob/main/examples/pytorch/llm/ds_config/zero3.json). 你只需要指定'default-zero2', 就会使用默认zero2配置文件; 指定'default-zero3', 就会使用默认的zero3配置文件.
 - `--batch_size`: 训练时的batch_size, 默认为`1`. 增大batch_size可以增加GPU的利用率, 但不一定会增加训练速度, 因为在一个batch中, 需要对较短的句子按该batch中最长句子的长度进行padding, 从而引入无效的计算量.
 - `--eval_batch_size`: 评估时的batch_size, 默认为`None`, 即当`predict_with_generate`为True时, 设置为1, 为False时, 设置为`batch_size`.
 - `--num_train_epochs`: 训练的epoch数, 默认为`1`. 如果`max_steps >= 0`, 则覆盖`num_train_epochs`. 通常情况下设置为3 ~ 5.
@@ -84,7 +84,7 @@
 - `--ignore_args_error`: 是否忽略命令行传参错误抛出的Error, 默认为`False`. 如果需要拷贝代码到notebook中运行, 需要设置成True.
 - `--check_model_is_latest`: 检查模型是否是最新, 默认为`True`. 如果你需要断网进行训练, 请将该参数设置为`False`.
 - `--logging_dir`: 默认为`None`. 即设置为`f'{self.output_dir}/runs'`, 表示tensorboard文件存储路径.
-- `--report_to`: 默认为`['all']`.
+- `--report_to`: 默认为`['tensorboard']`.
 - `--acc_strategy`: 默认为`'token'`, 可选择的值包括: 'token', 'sentence'.
 - `--save_on_each_node`: 该参数在多机训练时生效, 默认为`True`.
 - `--save_strategy`: 保存checkpoint的策略, 默认为`'steps'`, 可选择的值包括: 'steps', 'no'.
diff --git a/docs/source/LLM/支持的模型和数据集.md b/docs/source/LLM/支持的模型和数据集.md
@@ -177,6 +177,7 @@
 
 | Dataset Name | Dataset ID | Train Size | Val Size | Statistic (token) | Tags |
 | ------------ | ---------- | ---------- | -------- | ----------------- | ---- |
+|🔥ms-bench|[iic/ms_bench](https://modelscope.cn/datasets/iic/ms_bench/summary)|323733|0|343.4±441.2, min=22, max=30960|chat, general, multi-round|
 |🔥alpaca-en|[AI-ModelScope/alpaca-gpt4-data-en](https://modelscope.cn/datasets/AI-ModelScope/alpaca-gpt4-data-en/summary)|52002|0|176.2±125.8, min=26, max=740|chat, general|
 |🔥alpaca-zh|[AI-ModelScope/alpaca-gpt4-data-zh](https://modelscope.cn/datasets/AI-ModelScope/alpaca-gpt4-data-zh/summary)|48818|0|162.1±93.9, min=26, max=856|chat, general|
 |multi-alpaca-all|[damo/nlp_polylm_multialpaca_sft](https://modelscope.cn/datasets/damo/nlp_polylm_multialpaca_sft/summary)|131867|0|112.9±50.6, min=26, max=1226|chat, general, multilingual|
@@ -189,14 +190,15 @@
 |gpt4all-en|[wyj123456/GPT4all](https://modelscope.cn/datasets/wyj123456/GPT4all/summary)|806199|0|302.5±384.1, min=27, max=7391|chat, general|
 |sharegpt-en|[huangjintao/sharegpt](https://modelscope.cn/datasets/huangjintao/sharegpt/summary)|99799|0|1045.7±431.9, min=22, max=7907|chat, general, multi-round|
 |sharegpt-zh|[huangjintao/sharegpt](https://modelscope.cn/datasets/huangjintao/sharegpt/summary)|135399|0|806.3±771.7, min=21, max=65318|chat, general, multi-round|
-|tutu-v2-sft-mixture|[AI-ModelScope/tulu-v2-sft-mixture](https://modelscope.cn/datasets/AI-ModelScope/tulu-v2-sft-mixture/summary)|326154|0|867.8±996.4, min=22, max=12111|-|
+|🔥tulu-v2-sft-mixture|[AI-ModelScope/tulu-v2-sft-mixture](https://modelscope.cn/datasets/AI-ModelScope/tulu-v2-sft-mixture/summary)|326154|0|867.8±996.4, min=22, max=12111|chat, multilingual, general, multi-round|
 |wikipedia-zh|[AI-ModelScope/wikipedia-cn-20230720-filtered](https://modelscope.cn/datasets/AI-ModelScope/wikipedia-cn-20230720-filtered/summary)|254547|0|568.4±713.2, min=37, max=78678|text-generation, general, pretrained|
 |open-orca|[AI-ModelScope/OpenOrca](https://modelscope.cn/datasets/AI-ModelScope/OpenOrca/summary)|3239027|0|360.4±402.9, min=27, max=8672|chat, multilingual, general|
 |open-orca-gpt4|[AI-ModelScope/OpenOrca](https://modelscope.cn/datasets/AI-ModelScope/OpenOrca/summary)|994896|0|382.3±417.4, min=31, max=8740|chat, multilingual, general|
 |sharegpt-gpt4|[AI-ModelScope/sharegpt_gpt4](https://modelscope.cn/datasets/AI-ModelScope/sharegpt_gpt4/summary)|103063|0|1286.2±2089.4, min=22, max=221080|chat, multilingual, general, multi-round|
+|🔥ms-agent|[iic/ms_agent](https://modelscope.cn/datasets/iic/ms_agent/summary)|30000|0|647.7±217.1, min=199, max=2722|chat, agent, multi-round|
 |damo-agent-zh|[damo/MSAgent-Bench](https://modelscope.cn/datasets/damo/MSAgent-Bench/summary)|422115|161|965.7±440.9, min=321, max=31535|chat, agent, multi-round|
-|🔥damo-agent-mini-zh|[damo/MSAgent-Bench](https://modelscope.cn/datasets/damo/MSAgent-Bench/summary)|39964|152|1230.9±350.1, min=558, max=4982|chat, agent, multi-round|
-|🔥agent-instruct-all-en|[huangjintao/AgentInstruct_copy](https://modelscope.cn/datasets/huangjintao/AgentInstruct_copy/summary)|1866|0|1144.3±635.5, min=206, max=6412|chat, agent, multi-round|
+|damo-agent-mini-zh|[damo/MSAgent-Bench](https://modelscope.cn/datasets/damo/MSAgent-Bench/summary)|39964|152|1230.9±350.1, min=558, max=4982|chat, agent, multi-round|
+|agent-instruct-all-en|[huangjintao/AgentInstruct_copy](https://modelscope.cn/datasets/huangjintao/AgentInstruct_copy/summary)|1866|0|1144.3±635.5, min=206, max=6412|chat, agent, multi-round|
 |code-alpaca-en|[wyj123456/code_alpaca_en](https://modelscope.cn/datasets/wyj123456/code_alpaca_en/summary)|20016|0|100.1±60.1, min=29, max=1776|chat, coding|
 |🔥leetcode-python-en|[AI-ModelScope/leetcode-solutions-python](https://modelscope.cn/datasets/AI-ModelScope/leetcode-solutions-python/summary)|2359|0|723.8±233.5, min=259, max=2117|chat, coding|
 |🔥codefuse-python-en|[codefuse-ai/CodeExercise-Python-27k](https://modelscope.cn/datasets/codefuse-ai/CodeExercise-Python-27k/summary)|27224|0|483.6±193.9, min=45, max=3082|chat, coding|
diff --git a/examples/pytorch/llm/scripts/baichuan2_13b_chat/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/baichuan2_13b_chat/lora_ddp_ds/sft.sh
@@ -40,5 +40,5 @@ torchrun \
     --hub_model_id baichuan2-13b-chat-lora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/baichuan2_13b_chat/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/baichuan2_13b_chat/qlora_ddp_ds/sft.sh
@@ -42,5 +42,5 @@ torchrun \
     --hub_model_id baichuan2-13b-chat-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/baichuan2_13b_chat_int4/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/baichuan2_13b_chat_int4/qlora_ddp_ds/sft.sh
@@ -40,5 +40,5 @@ torchrun \
     --hub_model_id baichuan2-13b-chat-int4-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/baichuan2_7b_chat/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/baichuan2_7b_chat/lora_ddp_ds/sft.sh
@@ -40,5 +40,5 @@ torchrun \
     --hub_model_id baichuan2-7b-chat-lora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/baichuan2_7b_chat/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/baichuan2_7b_chat/qlora_ddp_ds/sft.sh
@@ -42,5 +42,5 @@ torchrun \
     --hub_model_id baichuan2-7b-chat-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/baichuan2_7b_chat_int4/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/baichuan2_7b_chat_int4/qlora_ddp_ds/sft.sh
@@ -40,5 +40,5 @@ torchrun \
     --hub_model_id baichuan2-7b-chat-int4-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/baichuan_13b_chat/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/baichuan_13b_chat/qlora_ddp_ds/sft.sh
@@ -42,5 +42,5 @@ torchrun \
     --hub_model_id baichuan-13b-chat-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/chatglm2_6b/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/chatglm2_6b/lora_ddp_ds/sft.sh
@@ -40,5 +40,5 @@ torchrun \
     --hub_model_id chatglm2-6b-lora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/chatglm3_6b/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/chatglm3_6b/lora_ddp_ds/sft.sh
@@ -40,5 +40,5 @@ torchrun \
     --hub_model_id chatglm3-6b-lora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/chatglm3_6b_base/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/chatglm3_6b_base/lora_ddp_ds/sft.sh
@@ -40,5 +40,5 @@ torchrun \
     --hub_model_id chatglm3-6b-base-lora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/codegeex2_6b/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/codegeex2_6b/lora_ddp_ds/sft.sh
@@ -39,5 +39,5 @@ torchrun \
     --hub_model_id codegeex2-6b-lora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/custom/tigerbot_13b_chat/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/custom/tigerbot_13b_chat/qlora_ddp_ds/sft.sh
@@ -41,5 +41,5 @@ torchrun \
     --hub_model_id tigerbot-13b-chat-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/custom/tigerbot_7b/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/custom/tigerbot_7b/lora_ddp_ds/sft.sh
@@ -39,5 +39,5 @@ torchrun \
     --hub_model_id tigerbot-13b-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/llama2_13b_chat/longlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/llama2_13b_chat/longlora_ddp_ds/sft.sh
@@ -40,5 +40,5 @@ torchrun \
     --hub_model_id llama2-13b-chat-longlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/llama2_13b_chat/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/llama2_13b_chat/qlora_ddp_ds/sft.sh
@@ -42,5 +42,5 @@ torchrun \
     --hub_model_id llama2-13b-chat-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/llama2_70b_chat/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/llama2_70b_chat/qlora_ddp_ds/sft.sh
@@ -42,5 +42,5 @@ torchrun \
     --hub_model_id llama2-70b-chat-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/mistral_7b_instruct/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/mistral_7b_instruct/lora_ddp_ds/sft.sh
@@ -40,5 +40,5 @@ torchrun \
     --hub_model_id mistral-7b-instruct-lora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/openbuddy_llama2_13b_chat/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/openbuddy_llama2_13b_chat/qlora_ddp_ds/sft.sh
@@ -42,5 +42,5 @@ torchrun \
     --hub_model_id openbuddy-llama2-13b-chat-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/openbuddy_llama2_70b_chat/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/openbuddy_llama2_70b_chat/qlora_ddp_ds/sft.sh
@@ -42,5 +42,5 @@ torchrun \
     --hub_model_id openbuddy-llama2-70b-chat-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/openbuddy_mistral_7b_chat/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/openbuddy_mistral_7b_chat/lora_ddp_ds/sft.sh
@@ -40,5 +40,5 @@ torchrun \
     --hub_model_id openbuddy-mistral-7b-chat-lora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/polylm_13b/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/polylm_13b/qlora_ddp_ds/sft.sh
@@ -42,5 +42,5 @@ torchrun \
     --hub_model_id polylm-13b-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/qwen_14b/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_14b/lora_ddp_ds/sft.sh
@@ -41,5 +41,5 @@ torchrun \
     --hub_model_id qwen-14b-lora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/qwen_14b/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_14b/qlora_ddp_ds/sft.sh
@@ -43,5 +43,5 @@ torchrun \
     --hub_model_id qwen-14b-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/qwen_14b_chat/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_14b_chat/lora_ddp_ds/sft.sh
@@ -41,5 +41,5 @@ torchrun \
     --hub_model_id qwen-14b-chat-lora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/qwen_14b_chat/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_14b_chat/qlora_ddp_ds/sft.sh
@@ -44,5 +44,5 @@ torchrun \
     --hub_model_id qwen-14b-chat-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/qwen_14b_chat_int4/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_14b_chat_int4/qlora_ddp_ds/sft.sh
@@ -41,5 +41,5 @@ torchrun \
     --hub_model_id qwen-14b-chat-int4-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/qwen_14b_chat_int8/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_14b_chat_int8/qlora_ddp_ds/sft.sh
@@ -41,5 +41,5 @@ torchrun \
     --hub_model_id qwen-14b-chat-int8-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/qwen_72b_chat_int4/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_72b_chat_int4/qlora_ddp_ds/sft.sh
@@ -42,6 +42,6 @@ torchrun \
     --hub_model_id qwen-72b-chat-int4-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
     --tuner_backend peft \
diff --git a/examples/pytorch/llm/scripts/qwen_72b_chat_int8/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_72b_chat_int8/qlora_ddp_ds/sft.sh
@@ -43,5 +43,5 @@ torchrun \
     --hub_model_id qwen-72b-chat-int8-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/qwen_7b/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_7b/lora_ddp_ds/sft.sh
@@ -41,5 +41,5 @@ torchrun \
     --hub_model_id qwen-7b-lora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/qwen_7b_chat/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_7b_chat/lora_ddp_ds/sft.sh
@@ -41,5 +41,5 @@ torchrun \
     --hub_model_id qwen-7b-chat-lora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/qwen_7b_chat/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_7b_chat/qlora_ddp_ds/sft.sh
@@ -44,5 +44,5 @@ torchrun \
     --hub_model_id qwen-7b-chat-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/qwen_7b_chat_int4/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_7b_chat_int4/qlora_ddp_ds/sft.sh
@@ -41,5 +41,5 @@ torchrun \
     --hub_model_id qwen-7b-chat-int4-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/qwen_7b_chat_int8/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_7b_chat_int8/qlora_ddp_ds/sft.sh
@@ -41,5 +41,5 @@ torchrun \
     --hub_model_id qwen-7b-chat-int8-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/qwen_audio_chat/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_audio_chat/lora_ddp_ds/sft.sh
@@ -40,6 +40,6 @@ torchrun \
     --hub_model_id qwen-audio-chat-lora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
     --lazy_tokenize true \
diff --git a/examples/pytorch/llm/scripts/qwen_vl/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_vl/lora_ddp_ds/sft.sh
@@ -41,5 +41,5 @@ torchrun \
     --hub_model_id qwen-vl-lora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/qwen_vl_chat/lora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_vl_chat/lora_ddp_ds/sft.sh
@@ -41,5 +41,5 @@ torchrun \
     --hub_model_id qwen-vl-chat-lora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/qwen_vl_chat_int4/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/qwen_vl_chat_int4/qlora_ddp_ds/sft.sh
@@ -41,5 +41,5 @@ torchrun \
     --hub_model_id qwen-vl-chat-qlora \
     --hub_private_repo true \
     --hub_token 'your-sdk-token' \
-    --deepspeed_config_path 'ds_config/zero2.json' \
+    --deepspeed_config_path default-zero2 \
     --save_only_model true \
diff --git a/examples/pytorch/llm/scripts/ziya2_13b_chat/qlora_ddp_ds/sft.sh b/examples/pytorch/llm/scripts/ziya2_13b_chat/qlora_ddp_ds/sft.sh
diff --git a/scripts/utils/run_dataset_info.py b/scripts/utils/run_dataset_info.py
diff --git a/swift/llm/utils/argument.py b/swift/llm/utils/argument.py
diff --git a/swift/llm/utils/dataset.py b/swift/llm/utils/dataset.py
diff --git a/swift/llm/utils/preprocess.py b/swift/llm/utils/preprocess.py
