Skip to content

Commit 9ed3674

Browse files
typhoonzerotyphoonzero
authored
Merge pull request #9729 from typhoonzero/update_release_doc
Update release doc
2 parents 65694db + f25825a commit 9ed3674

File tree

4 files changed

+242
-26
lines changed

4 files changed

+242
-26
lines changed

‎doc/fluid/dev/index_cn.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,5 +9,5 @@
99
use_eigen_cn.md
1010
name_convention.md
1111
support_new_device.md
12-
releasing_process.md
12+
releasing_process_cn.md
1313
op_markdown_format.md

‎doc/fluid/dev/index_en.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,5 +9,5 @@ Development
99
use_eigen_en.md
1010
name_convention.md
1111
support_new_device.md
12-
releasing_process.md
12+
releasing_process_en.md
1313
op_markdown_format.md

‎doc/fluid/dev/releasing_process.md renamed to ‎doc/fluid/dev/releasing_process_cn.md

Lines changed: 30 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -10,19 +10,10 @@ PaddlePaddle每次发新的版本，遵循以下流程:
1010
* 使用Regression Test List作为检查列表，测试本次release的正确性。
1111
* 如果失败，记录下所有失败的例子，在这个`release/版本号`分支中，修复所有bug后，Patch号加一，到第二步
1212
* 修改`python/setup.py.in`中的版本信息,并将`istaged`字段设为`True`。
13-
* 编译这个版本的python wheel包，并发布到pypi。
14-
* 由于pypi.python.org目前遵循[严格的命名规范PEP 513](https://www.python.org/dev/peps/pep-0513)，在使用twine上传之前，需要重命名wheel包中platform相关的后缀，比如将`linux_x86_64`修改成`manylinux1_x86_64`。
15-
* pypi上的package名称为paddlepaddle和paddlepaddle_gpu，如果要上传GPU版本的包，需要修改build/python/setup.py中，name: "paddlepaddle_gpu"并重新打包wheel包：`python setup.py bdist_wheel`。
16-
* 上传方法：
17-
```
18-
cd build/python
19-
pip install twine
20-
twine upload dist/[package to upload]
21-
```
22-
* 编译这个版本的Docker发行镜像，发布到dockerhub。如果失败，修复Docker编译镜像问题，Patch号加一，返回第二步
23-
1. 第三步完成后，将`release/版本号`分支合入master分支，并删除`release/版本号`分支。将master分支的合入commit打上tag，tag为`版本号`。同时再将`master`分支合入`develop`分支。最后删除`release/版本号`分支。
24-
1. 协同完成Release Note的书写
25-
13+
* 将这个版本的python wheel包发布到pypi。
14+
* 更新Docker镜像（参考后面的操作细节）。
15+
1. 第三步完成后，将`release/版本号`分支合入master分支，将master分支的合入commit打上tag，tag为`版本号`。同时再将`master`分支合入`develop`分支。
16+
1. 协同完成Release Note的书写。
2617

2718
需要注意的是:
2819

@@ -31,13 +22,18 @@ PaddlePaddle每次发新的版本，遵循以下流程:
3122

3223
## 发布wheel包到pypi
3324

34-
使用[PaddlePaddle CI](https://paddleci.ngrok.io/project.html?projectId=Manylinux1&tab=projectOverview)
25+
1. 使用[PaddlePaddle CI](https://paddleci.ngrok.io/project.html?projectId=Manylinux1&tab=projectOverview)
3526
完成自动化二进制编译，参考下图，选择需要发布的版本（通常包含一个CPU版本和一个GPU版本），点击"run"右侧的"..."按钮，可以
36-
弹出下面的选择框，在第二个tab (Changes)里选择需要发布的分支，这里选择0.11.0，然后点击"Run Build"按钮。等待编译完成后
37-
可以在此页面的"Artifacts"下拉框中找到生成的3个二进制文件，分别对应CAPI，`cp27m`和`cp27mu`的版本。然后按照上述的方法
38-
使用`twine`工具上传即可。
39-
40-
<img src="https://raw.githubusercontent.com/PaddlePaddle/Paddle/develop/doc/fluid/images/ci_build_whl.png">
27+
弹出下面的选择框，在第二个tab (Changes)里选择需要发布的分支，这里选择0.11.0，然后点击"Run Build"按钮。
28+
<img src="https://raw.githubusercontent.com/PaddlePaddle/Paddle/develop/doc/fluid/images/ci_build_whl.png">
29+
1. 等待编译完成后可以在此页面的"Artifacts"下拉框中找到生成的3个二进制文件，分别对应CAPI，`cp27m`和`cp27mu`的版本。
30+
1. 由于pypi.python.org目前遵循[严格的命名规范PEP 513](https://www.python.org/dev/peps/pep-0513)，在使用twine上传之前，需要重命名wheel包中platform相关的后缀，比如将`linux_x86_64`修改成`manylinux1_x86_64`。
31+
1. 上传：
32+
```
33+
cd build/python
34+
pip install twine
35+
twine upload dist/[package to upload]
36+
```
4137

4238
* 注：CI环境使用 https://github.com/PaddlePaddle/buildtools 这里的DockerImage作为编译环境以支持更多的Linux
4339
发型版，如果需要手动编译，也可以使用这些镜像。这些镜像也可以从 https://hub.docker.com/r/paddlepaddle/paddle_manylinux_devel/tags/ 下载得到。
@@ -48,10 +44,20 @@ PaddlePaddle每次发新的版本，遵循以下流程:
4844
上述PaddlePaddle CI编译wheel完成后会自动将Docker镜像push到DockerHub，所以，发布Docker镜像只需要对自动push的镜像打上
4945
版本号对应的tag即可：
5046

51-
1. 进入 https://hub.docker.com/r/paddlepaddle/paddle/tags/ 查看latest tag的更新时间是否在上述编译wheel包完成后是否最新。
52-
1. 执行 `docker pull paddlepaddle/paddle:[latest tag]`，latest tag可以是latest或latest-gpu等。
53-
1. 执行 `docker tag paddlepaddle/paddle:[latest tag] paddlepaddle/paddle:[version]`
54-
1. 执行 `docker push paddlepaddle/paddle:[version]`
47+
```
48+
docker pull [镜像]:latest
49+
docker tag [镜像]:latest [镜像]:[version]
50+
docker push [镜像]:[version]
51+
```
52+
53+
需要更新的镜像tag包括：
54+
55+
* `[version]`: CPU版本
56+
* `[version]-openblas`: openblas版本
57+
* `[version]-gpu`: GPU版本（CUDA 8.0 cudnn 5）
58+
* `[version]-gpu-[cudaver]-[cudnnver]`: 不同cuda, cudnn版本的镜像
59+
60+
之后可进入 https://hub.docker.com/r/paddlepaddle/paddle/tags/ 查看是否发布成功。
5561

5662
## PaddlePaddle 分支规范
5763

@@ -76,7 +82,7 @@ PaddlePaddle开发过程使用[git-flow](http://nvie.com/posts/a-successful-git-
7682

7783
### PaddlePaddle Book中所有章节
7884

79-
PaddlePaddle每次发版本首先要保证PaddlePaddle Book中所有章节功能的正确性。功能的正确性包括验证PaddlePaddle目前的`paddle_trainer`训练和纯使用`Python`训练模型正确性。
85+
PaddlePaddle每次发版本首先要保证PaddlePaddle Book中所有章节功能的正确性。功能的正确性包括验证PaddlePaddle目前的`paddle_trainer`训练和纯使用`Python`训练（V2和Fluid）模型正确性。
8086

8187
<table>
8288
<thead>

‎doc/fluid/dev/releasing_process_en.md

Lines changed: 210 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,210 @@
1+
# PaddlePaddle Releasing Process
2+
3+
PaddlePaddle manages its branches using "git-flow branching model", and [Semantic Versioning](http://semver.org/) as it's version number semantics.
4+
5+
Each time we release a new PaddlePaddle version, we should follow the below steps:
6+
7+
1. Fork a new branch from `develop` named `release/[version]`, e.g. `release/0.10.0`.
8+
1. Push a new tag on the release branch, the tag name should be like `[version]rc.patch`. The
9+
first tag should be `0.10.0rc1`, and the second should be `0.10.0.rc2` and so on.
10+
1. After that, we should do:
11+
* Run all regression test on the Regression Test List (see PaddlePaddle TeamCity CI), to confirm
12+
that this release has no major bugs.
13+
* If regression test fails, we must fix those bugs and create a new `release/[version]`
14+
branch from previous release branch.
15+
* Modify `python/setup.py.in`, change the version number and change `ISTAGED` to `True`.
16+
* Publish PaddlePaddle release wheel packages to pypi (see below instructions for detail).
17+
* Update the Docker images (see below instructions for detail).
18+
1. After above step, merge `release/[version]` branch to master and push a tag on the master commit,
19+
then merge `master` to `develop`.
20+
1. Update the Release Note.
21+
22+
***NOTE:***
23+
24+
* Do ***NOT*** merge commits from develop branch to release branches to keep the release branch contain
25+
features only for current release, so that we can test on that version.
26+
* If we want to fix bugs on release branches, we must merge the fix to master, develop and release branch.
27+
28+
## Publish Wheel Packages to pypi
29+
30+
1. Use our [CI tool](https://paddleci.ngrok.io/project.html?projectId=Manylinux1&tab=projectOverview)
31+
to build all wheel packages needed to publish. As shown in the following picture, choose a build
32+
version, click "..." button on the right side of "Run" button, and switch to the second tab in the
33+
pop-up box, choose the current release branch and click "Run Build" button. You may repeat this
34+
step to start different versions of builds.
35+
<img src="https://raw.githubusercontent.com/PaddlePaddle/Paddle/develop/doc/fluid/images/ci_build_whl.png">
36+
1. After the build succeeds, download the outputs under "Artifacts" including capi, `cp27m` and `cp27mu`.
37+
1. Since pypi.python.org follows [PEP 513](https://www.python.org/dev/peps/pep-0513), before we
38+
upload the package using `twine`, we need to rename the package from `linux_x86_64` to
39+
`manylinux1_x86_64`.
40+
1. Start the upload:
41+
```
42+
cd build/python
43+
pip install twine
44+
twine upload dist/[package to upload]
45+
```
46+
47+
* NOTE: We use a special Docker image to build our releases to support more Linux distributions, you can
48+
download it from https://hub.docker.com/r/paddlepaddle/paddle_manylinux_devel/tags/, or build it using
49+
scripts under `tools/manylinux1`.
50+
* pypi does not allow overwrite the already uploaded version of wheel package, even if you delete the
51+
old version. you must change the version number before upload a new one.
52+
53+
## Publish Docker Images
54+
55+
Our CI tool will push latest images to DockerHub, so we only need to push a version tag like:
56+
57+
```
58+
docker pull [image]:latest
59+
docker tag [image]:latest [image]:[version]
60+
docker push [image]:[version]
61+
```
62+
63+
Tags that need to be updated are:
64+
* `[version]`: CPU only version image
65+
* `[version]-openblas`: openblas version image
66+
* `[version]-gpu`: GPU version（using CUDA 8.0 cudnn 5）
67+
* `[version]-gpu-[cudaver]-[cudnnver]`: tag for different cuda, cudnn versions
68+
69+
You can then checkout the latest pushed tags at https://hub.docker.com/r/paddlepaddle/paddle/tags/.
70+
71+
## Branching Model
72+
73+
We use [git-flow](http://nvie.com/posts/a-successful-git-branching-model/) as our branching model,
74+
with some modifications:
75+
76+
* `master` branch is the stable branch. Each version on the master branch is tested and guaranteed.
77+
* `develop` branch is for development. Each commit on develop branch has passed CI unit test, but no
78+
regression tests are run.
79+
* `release/[version]` branch is used to publish each release. Latest release version branches have
80+
bugfix only for that version, but no feature updates.
81+
* Developer forks are not required to follow
82+
[git-flow](http://nvie.com/posts/a-successful-git-branching-model/)
83+
branching model, all forks is like a feature branch.
84+
* Advise: developer fork's develop branch is used to sync up with main repo's develop branch.
85+
* Advise: developer use it's fork's develop branch to for new branch to start developing.
86+
* Use that branch on developer's fork to create pull requests and start reviews.
87+
* developer can push new commits to that branch when the pull request is open.
88+
* Bug fixes are also started from developers forked repo. And, bug fixes branch can merge to
89+
`master`, `develop` and `releases`.
90+
91+
## PaddlePaddle Regression Test List
92+
93+
### All Chapters of PaddlePaddle Book
94+
95+
We need to guarantee that all the chapters of PaddlePaddle Book can run correctly. Including
96+
V1 (`paddle_trainer` training) and V2 training and Fluid training.
97+
98+
<table>
99+
<thead>
100+
<tr>
101+
<th></th>
102+
<th>Linear Regression</th>
103+
<th>Recognize Digits</th>
104+
<th>Image Classification</th>
105+
<th>Word2Vec</th>
106+
<th>Personalized Recommendation</th>
107+
<th>Sentiment Analysis</th>
108+
<th>Semantic Role Labeling</th>
109+
<th>Machine Translation</th>
110+
</tr>
111+
</thead>
112+
113+
<tbody>
114+
<tr>
115+
<td>API.V2 + Docker + GPU </td>
116+
<td> </td>
117+
<td> </td>
118+
<td> </td>
119+
<td> </td>
120+
<td> </td>
121+
<td> </td>
122+
<td> </td>
123+
<td> </td>
124+
</tr>
125+
126+
<tr>
127+
<td> API.V2 + Docker + CPU </td>
128+
<td> </td>
129+
<td> </td>
130+
<td> </td>
131+
<td> </td>
132+
<td> </td>
133+
<td> </td>
134+
<td> </td>
135+
<td> </td>
136+
</tr>
137+
138+
<tr>
139+
<td>`paddle_trainer` + Docker + GPU </td>
140+
<td> </td>
141+
<td> </td>
142+
<td> </td>
143+
<td> </td>
144+
<td> </td>
145+
<td> </td>
146+
<td> </td>
147+
<td> </td>
148+
</tr>
149+
150+
<tr>
151+
<td>`paddle_trainer` + Docker + CPU </td>
152+
<td> </td>
153+
<td> </td>
154+
<td> </td>
155+
<td> </td>
156+
<td> </td>
157+
<td> </td>
158+
<td> </td>
159+
<td> </td>
160+
</tr>
161+
162+
<tr>
163+
<td> API.V2 + Ubuntu + GPU</td>
164+
<td> </td>
165+
<td> </td>
166+
<td> </td>
167+
<td> </td>
168+
<td> </td>
169+
<td> </td>
170+
<td> </td>
171+
<td> </td>
172+
</tr>
173+
174+
<tr>
175+
<td>API.V2 + Ubuntu + CPU </td>
176+
<td> </td>
177+
<td> </td>
178+
<td> </td>
179+
<td> </td>
180+
<td> </td>
181+
<td> </td>
182+
<td> </td>
183+
<td> </td>
184+
</tr>
185+
186+
<tr>
187+
<td> `paddle_trainer` + Ubuntu + GPU</td>
188+
<td> </td>
189+
<td> </td>
190+
<td> </td>
191+
<td> </td>
192+
<td> </td>
193+
<td> </td>
194+
<td> </td>
195+
<td> </td>
196+
</tr>
197+
198+
<tr>
199+
<td> `paddle_trainer` + Ubuntu + CPU</td>
200+
<td> </td>
201+
<td> </td>
202+
<td> </td>
203+
<td> </td>
204+
<td> </td>
205+
<td> </td>
206+
<td> </td>
207+
<td> </td>
208+
</tr>
209+
</tbody>
210+
</table>

0 commit comments

Comments
 (0)