update docs (#3349)

Author: Jintao-Huang

README.md

@@ -393,7 +393,7 @@ CUDA_VISIBLE_DEVICES=0 swift export \
 
 ### Push Model
 ```shell
-CUDA_VISIBLE_DEVICES=0 swift export \
+swift export \
     --model <model-path> \
     --push_to_hub true \
     --hub_model_id '<model-id>' \

README_CN.md

@@ -381,7 +381,7 @@ CUDA_VISIBLE_DEVICES=0 swift export \
 
 ### 推送模型
 ```shell
-CUDA_VISIBLE_DEVICES=0 swift export \
+swift export \
     --model <model-path> \
     --push_to_hub true \
     --hub_model_id '<model-id>' \

docs/source/BestPractices/GRPO多模态训练.md

@@ -1,4 +1,4 @@
-# GRPO完整实验流程
+# 多模态GRPO完整实验流程
 本文介绍如何使用SWIFT GRPO进行多模态模型和任务的训练。目标是对多个多模态任务进行训练，提升任务精度，任务定义和训练参数等参考了 [R1-V](https://github.com/Deep-Agent/R1-V.git) 和 [open-r1-multimodal](https://github.com/EvolvingLMMs-Lab/open-r1-multimodal.git)
 
 

docs/source/Instruction/人类对齐.md

@@ -11,6 +11,8 @@ RM和DPO类算法如ORPO，CPO，SimPO，则需要 $(x,y_w,y_l)$ 格式的数据
 而KTO算法的数据比较特殊，只需要 $(x,y,\text{label})$ 格式的数据，其中 $x$ 表示模型输入，$y$ 表示模型输出，label表示回答是否符合人类偏好
 比如![kto_data](../../resources/kto_data.png)
 
+使用自定义数据集对文本模型或者多模态大模型进行RLHF训练可以参考[自定义数据集文档](../Customization/自定义数据集.md#rlhf)。
+
 ## GRPO
 [论文arvix](https://arxiv.org/abs/2402.03300)
 

docs/source/Instruction/导出.md

@@ -0,0 +1,55 @@
+# 导出与推送
+
+
+## Merge LoRA
+
+- 查看[这里](https://github.com/modelscope/ms-swift/blob/main/examples/export/merge_lora.sh)。
+
+## 量化
+
+SWFIT支持AWQ、GPTQ、BNB模型的量化导出。其中使用AWQ、GPTQ需使用校准数据集，量化性能较好但量化耗时较长；而BNB无需校准数据集，量化耗时较短。
+
+| 量化技术 | 多模态 | 推理加速 | 继续训练 |
+| -------- | ------ | -------- | -------- |
+| GPTQ     | ✅      | ✅        | ✅        |
+| AWQ      | ✅      | ✅        | ✅        |
+| BNB      | ❌      | ✅        | ✅        |
+
+
+除SWIFT安装外，需要安装以下额外依赖：
+```shell
+# 使用awq量化:
+# autoawq和cuda版本有对应关系，请按照`https://github.com/casper-hansen/AutoAWQ`选择版本
+# 如果出现torch依赖冲突，请额外增加指令`--no-deps`
+pip install autoawq -U
+
+# 使用gptq量化:
+# auto_gptq和cuda版本有对应关系，请按照`https://github.com/PanQiWei/AutoGPTQ#quick-installation`选择版本
+pip install auto_gptq optimum -U
+
+# 使用bnb量化：
+pip install bitsandbytes -U
+```
+
+我们提供了一系列脚本展现SWIFT的量化导出能力：
+- 支持[AWQ](https://github.com/modelscope/ms-swift/blob/main/examples/export/quantize/awq.sh)/[GPTQ](https://github.com/modelscope/ms-swift/blob/main/examples/export/quantize/gptq.sh)/[BNB](https://github.com/modelscope/ms-swift/blob/main/examples/export/quantize/bnb.sh)量化导出。
+- 多模态量化: 支持使用GPTQ和AWQ对多模态模型进行量化，其中AWQ支持的多模态模型有限。参考[这里](https://github.com/modelscope/ms-swift/tree/main/examples/export/quantize/mllm)。
+- 更多系列模型的支持: 支持[Bert](https://github.com/modelscope/ms-swift/tree/main/examples/export/quantize/bert)，[Reward Model](https://github.com/modelscope/ms-swift/tree/main/examples/export/quantize/reward_model)的量化导出。
+- 使用SWIFT量化导出的模型支持使用vllm/lmdeploy进行推理加速；也支持使用QLoRA继续进行SFT/RLHF。
+
+
+## 推送模型
+
+SWIFT支持将训练/量化的模型重新推送到ModelScope/HuggingFace。默认推送到ModelScope，你可以指定`--use_hf true`推送到HuggingFace。
+```shell
+swift export \
+    --model output/vx-xxx/checkpoint-xxx \
+    --push_to_hub true \
+    --hub_model_id '<model-id>' \
+    --hub_token '<sdk-token>' \
+    --use_hf false
+```
+
+小贴士：
+- 你可以使用`--model <checkpoint-dir>`或者`--adapters <checkpoint-dir>`指定需要推送的checkpoint目录，这两种写法在推送模型场景没有差异。
+- 推送到ModelScope时，你需要确保你已经注册了魔搭账号，你的SDK token可以在[该页面](https://www.modelscope.cn/my/myaccesstoken)中获取。推送模型需确保sdk token的账号具有model_id对应组织的编辑权限。推送模型将自动创建对应model_id的模型仓库（如果该模型仓库不存在），你可以使用`--hub_private_repo true`来自动创建私有的模型仓库。

docs/source/Instruction/导出与推送.md

@@ -106,7 +106,7 @@ result = sft_main(TrainArguments(
 
 ## Merge LoRA
 
-- 查看[这里](https://github.com/modelscope/ms-swift/blob/main/examples/export/merge_lora.sh)
+- 查看[这里](https://github.com/modelscope/ms-swift/blob/main/examples/export/merge_lora.sh)。
 
 ## 推理（微调后模型）
 

docs/source/Instruction/推送模型.md

@@ -23,13 +23,12 @@ Swift DOCUMENTATION
    Instruction/推理和部署.md
    Instruction/采样.md
    Instruction/评测.md
-   Instruction/导出.md
+   Instruction/导出与推送.md
    Instruction/强化微调.md
    Instruction/GRPO.md
    Instruction/支持的模型和数据集.md
    Instruction/使用tuners.md
    Instruction/智能体的支持.md
-   Instruction/推送模型.md
    Instruction/ReleaseNote3.0.md
    Instruction/常见问题整理.md
 

docs/source/Instruction/预训练与微调.md

@@ -0,0 +1,57 @@
+# Export and Push
+
+## Merge LoRA
+
+- See [here](https://github.com/modelscope/ms-swift/blob/main/examples/export/merge_lora.sh).
+
+## Quantization
+
+SWIFT supports quantization exports for AWQ, GPTQ, and BNB models. AWQ and GPTQ require a calibration dataset, which yields better quantization performance but takes longer to quantize. On the other hand, BNB does not require a calibration dataset and is quicker to quantize.
+
+| Quantization Technique | Multimodal | Inference Acceleration | Continued Training |
+| ---------------------- | ---------- | ---------------------- | ------------------ |
+| GPTQ                   | ✅          | ✅                      | ✅                  |
+| AWQ                    | ✅          | ✅                      | ✅                  |
+| BNB                    | ❌          | ✅                      | ✅                  |
+
+In addition to the SWIFT installation, the following additional dependencies need to be installed:
+
+```shell
+# For AWQ quantization:
+# The versions of autoawq and CUDA are correlated; please choose the version according to `https://github.com/casper-hansen/AutoAWQ`.
+# If there are dependency conflicts with torch, please add the `--no-deps` option.
+pip install autoawq -U
+
+# For GPTQ quantization:
+# The versions of auto_gptq and CUDA are correlated; please choose the version according to `https://github.com/PanQiWei/AutoGPTQ#quick-installation`.
+pip install auto_gptq optimum -U
+
+# For BNB quantization:
+pip install bitsandbytes -U
+```
+
+We provide a series of scripts to demonstrate SWIFT's quantization export capabilities:
+
+- Supports [AWQ](https://github.com/modelscope/ms-swift/blob/main/examples/export/quantize/awq.sh)/[GPTQ](https://github.com/modelscope/ms-swift/blob/main/examples/export/quantize/gptq.sh)/[BNB](https://github.com/modelscope/ms-swift/blob/main/examples/export/quantize/bnb.sh) quantization exports.
+- Multimodal quantization: Supports quantizing multimodal models using GPTQ and AWQ, with limited multimodal models supported by AWQ. Refer to [here](https://github.com/modelscope/ms-swift/tree/main/examples/export/quantize/mllm).
+- Support for more model series: Supports quantization exports for [BERT](https://github.com/modelscope/ms-swift/tree/main/examples/export/quantize/bert) and [Reward Model](https://github.com/modelscope/ms-swift/tree/main/examples/export/quantize/reward_model).
+- Models exported with SWIFT's quantization support inference acceleration using vllm/lmdeploy; they also support further SFT/RLHF using QLoRA.
+
+
+## Push Model
+
+SWIFT supports re-pushing trained/quantized models to ModelScope/Hugging Face. By default, it pushes to ModelScope, but you can specify `--use_hf true` to push to Hugging Face.
+
+```shell
+swift export \
+    --model output/vx-xxx/checkpoint-xxx \
+    --push_to_hub true \
+    --hub_model_id '<model-id>' \
+    --hub_token '<sdk-token>' \
+    --use_hf false
+```
+
+Tips:
+
+- You can use `--model <checkpoint-dir>` or `--adapters <checkpoint-dir>` to specify the checkpoint directory to be pushed. There is no difference between these two methods in the model pushing scenario.
+- When pushing to ModelScope, you need to make sure you have registered for a ModelScope account. Your SDK token can be obtained from [this page](https://www.modelscope.cn/my/myaccesstoken). Ensure that the account associated with the SDK token has edit permissions for the organization corresponding to the model_id. The model pushing process will automatically create a model repository corresponding to the model_id (if it does not already exist), and you can use `--hub_private_repo true` to automatically create a private model repository.