Skip to content

[Docathon][Fix Doc Format No.10] fix global_gather_cn #7480

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: develop
Choose a base branch
from
+0 −50
Open

[Docathon][Fix Doc Format No.10] fix global_gather_cn #7480

Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
3bb805e
feat: 文档修复
aztice Sep 26, 2025
e2e5a0b
feat: 格式化Python代码
aztice Sep 26, 2025
6bc70c7
remove: code block
aztice Sep 27, 2025
f51cb2d
remove: 异常 COPY-FROM
aztice Sep 27, 2025
cc3ec93
feat: 更新代码
aztice Sep 28, 2025
16a71eb
remove: useless word(s)/symbol(s)
aztice Sep 28, 2025
55ab602
remove: code block
aztice Sep 30, 2025
64bfbf4
Delete docs/api/paddle/distributed/utils/global_gather_cn.rst
aztice Sep 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions docs/api/paddle/distributed/utils/global_gather_cn.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ global_gather 根据 global_count 将 x 的数据收集到 n_expert * world_size
如下图所示，world_size 是 2，n_expert 是 2，x 的 batch_size 是 4，local_count 是[2, 0, 2, 0]，0 卡的 global_count 是[2, 0, , ],
1 卡的 global_count 是[2, 0, ,](因为篇幅问题，这里只展示在 0 卡运算的数据)，在 global_gather 算子里，
global_count 和 local_count 的意义与其在 global_scatter 里正好相反，
global_count[i]代表向第 (i // n_expert)张卡的第 (i % n_expert)个 expert 发送 local_expert[i]个数据，
local_count[i]代表从第 (i // n_expert)张卡接收 global_count[i]个数据给本卡的 第(i % n_expert)个 expert。
global_count[i] 代表向第 (i // n_expert)张卡的第 (i % n_expert)个 expert 发送 local_expert[i]个数据，
local_count[i] 代表从第 (i // n_expert)张卡接收 global_count[i] 个数据给本卡的 第(i % n_expert)个 expert。
发送的数据会按照每张卡的每个 expert 排列。图中的 rank0 代表第 0 张卡，rank1 代表第 1 张卡。

global_gather 发送数据的流程如下：
Expand Down Expand Up @@ -43,8 +43,9 @@ global_gather 发送数据的流程如下：

返回
:::::::::
Tensor，从所有 expert 接收的数据。
Tensor，从所有 expert 接收的数据喵~

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这句不用改哈~

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不能留着嘛QAQ

Copy link
Contributor

@Echo-Nie Echo-Nie Sep 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

呐，不能留着喵～ 只需要修改有误的地方，方便用户看懂 QAQ

代码示例
:::::::::

COPY-FROM: paddle.distributed.utils.global_gather
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里目前渲染结果如下：
image

正常的COPY-FROM应该是和英文api源码对齐，上面的渲染是一个bug；需要检索一下该api的路径位置可能是发生了改变，所以可能需要变更路径~
可以参考 #7472 的检索方法，如果无法检索该API的话就可以考虑删除~

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这边看了下，我最新的3.2版本的文档并没有这个api，旧版可以找到，估计是被移除了。

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/PaddlePaddle/Paddle/blob/2588f4899106cd27bdfcc84ba4c2f5f7aac570ab/python/paddle/distributed/utils/moe_utils.py#L154