Skip to content

Commit 66d9cf4

Browse files
committed
update qwen2 docs (#1097)
1 parent 9858e4f commit 66d9cf4

13 files changed

+315
-218
lines changed

docs/source/LLM/Qwen1.5全流程最佳实践.md

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -193,8 +193,7 @@ sft_args = SftArguments(
193193
f'{DatasetName.self_cognition}#500'],
194194
logging_steps=5,
195195
max_length=2048,
196-
learning_rate=5e-5,
197-
warmup_ratio=0.4,
196+
learning_rate=1e-4,
198197
output_dir='output',
199198
lora_target_modules=['ALL'],
200199
model_name=['小黄', 'Xiao Huang'],
@@ -216,8 +215,7 @@ swift sft \
216215
--dataset alpaca-zh#500 alpaca-en#500 self-cognition#500 \
217216
--logging_steps 5 \
218217
--max_length 2048 \
219-
--learning_rate 5e-5 \
220-
--warmup_ratio 0.4 \
218+
--learning_rate 1e-4 \
221219
--output_dir output \
222220
--lora_target_modules ALL \
223221
--model_name 小黄 'Xiao Huang' \
@@ -235,8 +233,7 @@ swift sft \
235233
--dataset alpaca-zh#500 alpaca-en#500 self-cognition#500 \
236234
--logging_steps 5 \
237235
--max_length 2048 \
238-
--learning_rate 5e-5 \
239-
--warmup_ratio 0.4 \
236+
--learning_rate 1e-4 \
240237
--output_dir output \
241238
--lora_target_modules ALL \
242239
--model_name 小黄 'Xiao Huang' \
@@ -486,8 +483,7 @@ swift sft \
486483
--dataset alpaca-zh#500 alpaca-en#500 self-cognition#500 \
487484
--logging_steps 5 \
488485
--max_length 4096 \
489-
--learning_rate 5e-5 \
490-
--warmup_ratio 0.4 \
486+
--learning_rate 1e-4 \
491487
--output_dir output \
492488
--lora_target_modules ALL \
493489
--model_name 小黄 'Xiao Huang' \

docs/source/LLM/VLLM推理加速与部署.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -530,8 +530,7 @@ swift sft \
530530
--dataset self-cognition#500 sharegpt-gpt4:default#1000 \
531531
--logging_steps 5 \
532532
--max_length 4096 \
533-
--learning_rate 5e-5 \
534-
--warmup_ratio 0.4 \
533+
--learning_rate 1e-4 \
535534
--output_dir output \
536535
--lora_target_modules ALL \
537536
--model_name 小黄 'Xiao Huang' \

docs/source/LLM/命令行参数.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@
104104
- `--ignore_args_error`: 是否忽略命令行传参错误抛出的Error, 默认为`False`. 如果需要拷贝代码到notebook中运行, 需要设置成True.
105105
- `--check_model_is_latest`: 检查模型是否是最新, 默认为`True`. 如果你需要断网进行训练, 请将该参数设置为`False`.
106106
- `--logging_dir`: 默认为`None`. 即设置为`f'{self.output_dir}/runs'`, 表示tensorboard文件存储路径.
107-
- `--report_to`: 默认为`['tensorboard']`.
107+
- `--report_to`: 默认为`['tensorboard']`. 可以设置`--report_to all`来报告所有已安装的集成.
108108
- `--acc_strategy`: 默认为`'token'`, 可选择的值包括: 'token', 'sentence'.
109109
- `--save_on_each_node`: 该参数在多机训练时生效, 默认为`True`.
110110
- `--save_strategy`: 保存checkpoint的策略, 默认为`'steps'`, 可选择的值包括: 'steps', 'epoch', 'no'.

docs/source/LLM/自我认知微调最佳实践.md

Lines changed: 124 additions & 71 deletions
Large diffs are not rendered by default.

docs/source_en/LLM/Command-line-parameters.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@
104104
- `--ignore_args_error`: Whether to ignore Error thrown by command line parameter errors, default is `False`. Set to True if need to copy code to notebook to run.
105105
- `--check_model_is_latest`: Check if model is latest, default is `True`. Set this to `False` if you need to train offline.
106106
- `--logging_dir`: Default is `None`. I.e. set to `f'{self.output_dir}/runs'`, representing path to store tensorboard files.
107-
- `--report_to`: Default is `['tensorboard']`.
107+
- `--report_to`: Default is `['tensorboard']`. You can set `--report_to all` to report to all installed integrations.
108108
- `--acc_strategy`: Default is `'token'`, options include: 'token', 'sentence'.
109109
- `--save_on_each_node`: Takes effect during multi-machine training, default is `True`.
110110
- `--save_strategy`: Strategy for saving checkpoint, default is `'steps'`, options include: 'steps', 'epoch', no'.

docs/source_en/LLM/Qwen1.5-best-practice.md

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -191,8 +191,7 @@ sft_args = SftArguments(
191191
f'{DatasetName.self_cognition}#500'],
192192
logging_steps=5,
193193
max_length=2048,
194-
learning_rate=5e-5,
195-
warmup_ratio=0.4,
194+
learning_rate=1e-4,
196195
output_dir='output',
197196
lora_target_modules=['ALL'],
198197
model_name=['小黄', 'Xiao Huang'],
@@ -215,8 +214,7 @@ swift sft \
215214
--dataset alpaca-zh#500 alpaca-en#500 self-cognition#500 \
216215
--logging_steps 5 \
217216
--max_length 2048 \
218-
--learning_rate 5e-5 \
219-
--warmup_ratio 0.4 \
217+
--learning_rate 1e-4 \
220218
--output_dir output \
221219
--lora_target_modules ALL \
222220
--model_name 小黄 'Xiao Huang' \
@@ -234,8 +232,7 @@ swift sft \
234232
--dataset alpaca-zh#500 alpaca-en#500 self-cognition#500 \
235233
--logging_steps 5 \
236234
--max_length 2048 \
237-
--learning_rate 5e-5 \
238-
--warmup_ratio 0.4 \
235+
--learning_rate 1e-4 \
239236
--output_dir output \
240237
--lora_target_modules ALL \
241238
--model_name 小黄 'Xiao Huang' \
@@ -488,8 +485,7 @@ swift sft \
488485
--dataset alpaca-zh#500 alpaca-en#500 self-cognition#500 \
489486
--logging_steps 5 \
490487
--max_length 4096 \
491-
--learning_rate 5e-5 \
492-
--warmup_ratio 0.4 \
488+
--learning_rate 1e-4 \
493489
--output_dir output \
494490
--lora_target_modules ALL \
495491
--model_name 小黄 'Xiao Huang' \

0 commit comments

Comments
 (0)