Skip to content

Commit 830f622

Browse files
committed
[docs] update docs (#6180)
1 parent c27dd1a commit 830f622

File tree

6 files changed

+6
-6
lines changed

6 files changed

+6
-6
lines changed

docs/source/Customization/自定义数据集.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -206,7 +206,7 @@ alpaca格式:
206206
- 不同模型grounding任务的特殊字符和数据集格式不同。
207207
- 不同模型对bbox是否归一化的处理不同。例如:qwen2.5-vl使用绝对坐标,而qwen2/3-vl、internvl2.5需要对bbox的坐标进行千分位坐标归一化。
208208
- 注意:Qwen2.5-VL采用绝对坐标,因此要小心每次的图像缩放,如果使用方案一的数据集格式,你需要预先对图像进行resize(H和W需要是28的系数),并根据该尺寸缩放坐标点。如果使用方案二的数据集格式,ms-swift会帮助你处理图像的缩放问题,你依旧可以使用`MAX_PIXELS`或者`--max_pixels`等进行图像缩放(仅训练,推理场景,你依旧需要自己处理图像的缩放问题)。
209-
- 对于Qwen2.5-VL/Qwen3-VL,你可以使用环境`QWENVL_BBOX_FORMAT='new'`(默认为'legacy'),以兼容[官方cookbook](https://github.com/QwenLM/Qwen3-VL/blob/main/cookbooks/2d_grounding.ipynb)格式。并将数据集定义成以下格式:
209+
- 对于Qwen2.5-VL/Qwen3-VL,你可以使用环境`QWENVL_BBOX_FORMAT='new'`(默认为'legacy',需"ms-swift>=3.9.1"),以兼容[官方cookbook](https://github.com/QwenLM/Qwen3-VL/blob/main/cookbooks/2d_grounding.ipynb)格式。并将数据集定义成以下格式:
210210
```jsonl
211211
{"messages": [{"role": "user", "content": "<image>找到图像中的<ref-object>"}, {"role": "assistant", "content": "[\n\t{\"bbox_2d\": <bbox>, \"label\": \"<ref-object>\"}\n\t{\"bbox_2d\": <bbox>, \"label\": \"<ref-object>\"}\n]"}], "images": ["cat.png"], "objects": {"ref": ["", "", ""], "bbox": [[90.9, 160.8, 135, 212.8], [360.9, 480.8, 495, 532.8]]}}
212212
```

docs/source/Instruction/命令行参数.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -715,7 +715,7 @@ App参数继承于[部署参数](#部署参数), [Web-UI参数](#Web-UI参数)
715715
- FPS: 默认为2.0。
716716
- FPS_MIN_FRAMES: 默认为4。一段视频的最小抽帧数。
717717
- 🔥FPS_MAX_FRAMES: 默认为768。一段视频的最大抽帧数。
718-
- QWENVL_BBOX_FORMAT: grounding格式使用'legacy'还是'new'。'legacy'格式为:`<|object_ref_start|>一只狗<|object_ref_end|><|box_start|>(432,991),(1111,2077)<|box_end|>`,'new'格式参考:[Qwen3-VL cookbook](https://github.com/QwenLM/Qwen3-VL/blob/main/cookbooks/2d_grounding.ipynb),并参考[grounding数据集格式文档](../Customization/自定义数据集.md#grounding)。默认为'legacy'。
718+
- 🔥QWENVL_BBOX_FORMAT: (ms-swift>=3.9.1) grounding格式使用'legacy'还是'new'。'legacy'格式为:`<|object_ref_start|>一只狗<|object_ref_end|><|box_start|>(432,991),(1111,2077)<|box_end|>`,'new'格式参考:[Qwen3-VL cookbook](https://github.com/QwenLM/Qwen3-VL/blob/main/cookbooks/2d_grounding.ipynb),并参考[grounding数据集格式文档](../Customization/自定义数据集.md#grounding)。默认为'legacy'。
719719
- 注意:该环境变量适配Qwen2/2.5/3-VL和Qwen2.5/3-Omni系列模型。
720720

721721
### qwen2_audio

docs/source/Megatron-SWIFT/命令行参数.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
- 🔥recompute_granularity: 重新计算激活的粒度,可选项为'full', 'selective'。其中full代表重新计算整个transformer layer,selective代表只计算transformer layer中的核心注意力部分。通常'selective'是推荐的。默认为'selective'。
99
- 当你设置为'selective'时,你可以通过指定`--recompute_modules`来选择对哪些部分进行重新计算。
1010
- 🔥recompute_method: 该参数需将recompute_granularity设置为'full'才生效,可选项为'uniform', 'block'。默认为None。
11-
- 🔥recompute_num_layers: 该参数需将recompute_granularity设置为'full'才生效,默认为None。若`recompute_method`设置为uniform,该参数含义为每个均匀划分的重新计算单元的transformer layers数量。例如你可以指定为`--recompute_granularity full --recompute_method uniform --recompute_num_layers 4`。recompute_num_layers越大,显存占用越小,计算成本越大。默认为None。
11+
- 🔥recompute_num_layers: 该参数需将recompute_granularity设置为'full'才生效,默认为None。若`recompute_method`设置为uniform,该参数含义为每个均匀划分的重新计算单元的transformer layers数量。例如你可以指定为`--recompute_granularity full --recompute_method uniform --recompute_num_layers 4`。recompute_num_layers越大,显存占用越小,计算成本越大。注意:当前进程中的模型层数需能被`recompute_num_layers`整除。默认为None。
1212
- 🔥recompute_modules: 选项包括"core_attn", "moe_act", "layernorm", "mla_up_proj", "mlp", "moe",默认值为`["core_attn"]`。该参数在`--recompute_granularity selective`时生效。例如在MoE训练时,你可以通过指定`--recompute_granularity selective --recompute_modules core_attn moe`降低显存。其中"core_attn"、"mlp" 和 "moe" 使用常规检查点,"moe_act"、"layernorm" 和 "mla_up_proj" 使用输出丢弃检查点。
1313
- "core_attn":重新计算 Transformer 层中的核心注意力部分。
1414
- "mlp":重新计算密集的 MLP 层。

docs/source_en/Customization/Custom-dataset.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -218,7 +218,7 @@ When using this type of data, please note:
218218
- Different models have different special characters and data format for the grounding task.
219219
- The handling of bounding box normalization varies across different models: for example, qwen2.5-vl uses absolute coordinates, while qwen2/3-vl and internvl2.5 require bounding box coordinates to be normalized to the thousandth scale.
220220
- Note: Qwen2.5-VL uses absolute coordinates, so you need to be careful with image resizing each time. If you use the dataset format from Option 1, you need to resize the images in advance (height and width must be multiples of 28) and scale the coordinates accordingly. If you use the dataset format from Option 2, ms-swift will handle image resizing for you. You can still use `MAX_PIXELS` or `--max_pixels` for image resizing (training only; for inference, you still need to handle image resizing yourself).
221-
- For Qwen2.5-VL/Qwen3-VL, you can set the environment variable `QWENVL_BBOX_FORMAT='new'` (default is `'legacy'`) to be compatible with the [official cookbook](https://github.com/QwenLM/Qwen3-VL/blob/main/cookbooks/2d_grounding.ipynb) format. Define your dataset in the following format:
221+
- For Qwen2.5-VL/Qwen3-VL, you can set the environment variable `QWENVL_BBOX_FORMAT='new'` (default is `'legacy'`, requires "ms-swift>=3.9.1") to be compatible with the [official cookbook](https://github.com/QwenLM/Qwen3-VL/blob/main/cookbooks/2d_grounding.ipynb) format. Define your dataset in the following format:
222222
```jsonl
223223
{"messages": [{"role": "user", "content": "<image>Locate the <ref-object> in the image"}, {"role": "assistant", "content": "[\n\t{\"bbox_2d\": <bbox>, \"label\": \"<ref-object>\"}\n\t{\"bbox_2d\": <bbox>, \"label\": \"<ref-object>\"}\n]"}], "images": ["cat.png"], "objects": {"ref": ["sheep", "sheep", "sheep"], "bbox": [[90.9, 160.8, 135, 212.8], [360.9, 480.8, 495, 532.8]]}}
224224
```

docs/source_en/Instruction/Command-line-parameters.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -735,7 +735,7 @@ These parameters have the same meaning as in `qwen_vl_utils<0.0.12` or the `qwen
735735
- FPS: Default is 2.0.
736736
- FPS_MIN_FRAMES: Default is 4. Minimum number of frames extracted from a video clip.
737737
- 🔥FPS_MAX_FRAMES: Default is 768. Maximum number of frames extracted from a video clip.
738-
- QWENVL_BBOX_FORMAT : Specifies whether to use `'legacy'` or `'new'` format for grounding. The `'legacy'` format is: `<|object_ref_start|>a dog<|object_ref_end|><|box_start|>(432,991),(1111,2077)<|box_end|>`. The `'new'` format refers to: [Qwen3-VL Cookbook](https://github.com/QwenLM/Qwen3-VL/blob/main/cookbooks/2d_grounding.ipynb). For dataset formatting, see the [Grounding Dataset Format Documentation](../Customization/Custom-dataset.md#grounding). Default: `'legacy'`.
738+
- 🔥QWENVL_BBOX_FORMAT: (ms-swift>=3.9.1) Specifies whether to use `'legacy'` or `'new'` format for grounding. The `'legacy'` format is: `<|object_ref_start|>a dog<|object_ref_end|><|box_start|>(432,991),(1111,2077)<|box_end|>`. The `'new'` format refers to: [Qwen3-VL Cookbook](https://github.com/QwenLM/Qwen3-VL/blob/main/cookbooks/2d_grounding.ipynb). For dataset formatting, see the [Grounding Dataset Format Documentation](../Customization/Custom-dataset.md#grounding). Default: `'legacy'`.
739739
- Note: This environment variable applies to Qwen2/2.5/3-VL and Qwen2.5/3-Omni series models.
740740

741741
### qwen2_audio

docs/source_en/Megatron-SWIFT/Command-line-parameters.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
- 🔥recompute_granularity: Granularity of activation recomputation, options are 'full', 'selective'. 'full' means recomputing the entire transformer layer, while 'selective' means only recomputing the core attention part of the transformer layer. 'selective' is generally recommended. Default is 'selective'.
1010
- When you set it to 'selective', you can specify `--recompute_modules` to choose which parts to recompute.
1111
- 🔥recompute_method: This parameter takes effect only when recompute_granularity is set to 'full', options are 'uniform', 'block'. Default is None.
12-
- 🔥recompute_num_layers: This parameter takes effect only when recompute_granularity is set to 'full'. Default is None. If `recompute_method` is set to uniform, this parameter specifies the number of transformer layers in each uniformly divided recomputation unit. For example, you can specify `--recompute_granularity full --recompute_method uniform --recompute_num_layers 4`. The larger the recompute_num_layers, the smaller the memory usage but higher computation cost. Default is None.
12+
- 🔥recompute_num_layers: This parameter takes effect only when recompute_granularity is set to 'full'. Default is None. If `recompute_method` is set to uniform, this parameter specifies the number of transformer layers in each uniformly divided recomputation unit. For example, you can specify `--recompute_granularity full --recompute_method uniform --recompute_num_layers 4`. The larger the recompute_num_layers, the smaller the memory usage but higher computation cost. Note: The number of model layers in the current process must be divisible by `recompute_num_layers`. Default is None.
1313
- 🔥recompute_modules: Options include "core_attn", "moe_act", "layernorm", "mla_up_proj", "mlp", and "moe". The default value is `["core_attn"]`. This parameter takes effect when `--recompute_granularity selective` is set. For example, during MoE training, you can reduce memory usage by specifying `--recompute_granularity selective --recompute_modules core_attn moe`. Among these, "core_attn", "mlp", and "moe" use normal checkpointing, while "moe_act", "layernorm", and "mla_up_proj" use output-discarding checkpointing.
1414
- "core_attn": Recomputes the core attention part of the Transformer layer.
1515
- "mlp": Recomputes the dense MLP layer.

0 commit comments

Comments
 (0)