You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: paddle/scripts/docker/README.md
+21-21Lines changed: 21 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,32 +2,32 @@
2
2
3
3
## Goals
4
4
5
-
We want the building procedure generates Docker images, so we can run PaddlePaddle applications on Kubernetes clusters.
5
+
We want the building procedure generates Docker images so that we can run PaddlePaddle applications on Kubernetes clusters.
6
6
7
-
We want it generates .deb packages, so that enterprises without Docker support can run PaddlePaddle applications as well.
7
+
We want to build .deb packages so that enterprise users can run PaddlePaddle applications without Docker.
8
8
9
-
We want to minimize the size of generated Docker images and .deb packages so to ease the deployment cost.
9
+
We want to minimize the size of generated Docker images and .deb packages so to reduce the download time.
10
10
11
11
We want to encapsulate building tools and dependencies in a *development* Docker image so to ease the tools installation for developers.
12
12
13
-
We want developers can use whatever editing tools (emacs, vim, Eclipse, Jupyter Notebook), so the development Docker image contains only building tools, not editing tools, and developers are supposed to git clone source code into their development computers, instead of the container running the development Docker image.
13
+
Developers use various editors (emacs, vim, Eclipse, Jupyter Notebook), so the development Docker image contains only building tools, not editing tools, and developers are supposed to git clone source code into their development computers and map the code into the development container.
14
14
15
-
We want the procedure and tools work also with testing, continuous integration, and releasing.
15
+
We want the procedure and tools also work with testing, continuous integration, and releasing.
16
16
17
17
18
18
## Docker Images
19
19
20
-
We want two Docker images for each version of PaddlePaddle:
20
+
So we need two Docker images for each version of PaddlePaddle:
21
21
22
22
1.`paddle:<version>-dev`
23
23
24
-
This a development image contains only the development tools. This standardizes the building tools and procedure. Users include:
24
+
This a development image contains only the development tools and standardizes the building procedure. Users include:
25
25
26
26
- developers -- no longer need to install development tools on the host, and can build their current work on the host (development computer).
27
27
- release engineers -- use this to build the official release from certain branch/tag on Github.com.
28
28
- document writers / Website developers -- Our documents are in the source repo in the form of .md/.rst files and comments in source code. We need tools to extract the information, typeset, and generate Web pages.
29
29
30
-
Of course developers can install building tools on their development computers. But different version of PaddlePaddle might require different set/version of building tools. Also, it makes collaborative debugging eaiser if all developers use a unified development environment.
30
+
Of course, developers can install building tools on their development computers. But different versions of PaddlePaddle might require different set or version of building tools. Also, it makes collaborative debugging easier if all developers use a unified development environment.
31
31
32
32
The development image should include the following tools:
33
33
@@ -38,7 +38,7 @@ We want two Docker images for each version of PaddlePaddle:
38
38
- woboq
39
39
- sshd
40
40
41
-
where `sshd` makes it easy for developers to have multiple terminals connecting into the container. `docker exec`works too, but if the container is running on a remote machine, it would be easier to ssh directly into the container than ssh to the box and run `docker exec`.
41
+
Many developers work on a remote computer with GPU; they could ssh into the computer and `docker exec`into the development container. However, running `sshd` in the container allows developers to ssh into the container directly.
42
42
43
43
1.`paddle:<version>`
44
44
@@ -49,9 +49,9 @@ We want two Docker images for each version of PaddlePaddle:
49
49
- no-GPU/AVX `paddle:<version>`
50
50
- no-GPU/no-AVX `paddle:<version>-noavx`
51
51
52
-
We'd like to give users the choice between GPU and no-GPU, because the GPU version image is much larger than then the no-GPU version.
52
+
We allow users to choose between GPU and no-GPU because the GPU version image is much larger than then the no-GPU version.
53
53
54
-
We'd like to give users the choice between AVX and no-AVX, because some cloud providers don't provide AVX-enabled VMs.
54
+
We allow users the choice between AVX and no-AVX, because some cloud providers don't provide AVX-enabled VMs.
55
55
56
56
57
57
## Development Environment
@@ -60,28 +60,28 @@ Here we describe how to use above two images. We start from considering our dai
60
60
61
61
Developers work on a computer, which is usually a laptop or desktop:
A basic principle is that source code lies on the development computer (host), so that editing tools like Eclipse can parse the source code and support auto-completion.
69
+
A principle here is that source code lies on the development computer (host) so that editors like Eclipse can parse the source code to support auto-completion.
70
70
71
71
72
72
## Usages
73
73
74
74
### Build the Development Docker Image
75
75
76
-
The following commands check out the source code on the development computer (host) and build the development image `paddle:dev`:
76
+
The following commands check out the source code to the host and build the development image `paddle:dev`:
The `docker build` command assumes that `Dockerfile` is in the root source tree. This is reasonable because this Dockerfile is this only on in our repo in this design.
84
+
The `docker build` command assumes that `Dockerfile` is in the root source tree. Note that in this design, this `Dockerfile` is this only one in our repo.
85
85
86
86
87
87
### Build PaddlePaddle from Source Code
@@ -92,7 +92,7 @@ Given the development image `paddle:dev`, the following command builds PaddlePad
This command mounts the source directory on the host into `/paddle` in the container, so the default entrypoint of `paddle:dev`, `build.sh`, would build the source code with possible local changes. When it writes to `/paddle/build` in the container, it actually writes to `$PWD/build` on the host.
95
+
This command mounts the source directory on the host into `/paddle` in the container, so the default entry point of `paddle:dev`, `build.sh`, could build the source code with possible local changes. When it writes to `/paddle/build` in the container, it writes to `$PWD/build` on the host indeed.
96
96
97
97
`build.sh` builds the following:
98
98
@@ -109,19 +109,19 @@ The following command builds the production image:
109
109
docker build -t paddle -f build/Dockerfile .
110
110
```
111
111
112
-
This production image is minimal -- it includes binary `paddle`, the share library `libpaddle.so`, and Python runtime.
112
+
This production image is minimal -- it includes binary `paddle`, the shared library `libpaddle.so`, and Python runtime.
113
113
114
114
### Run PaddlePaddle Applications
115
115
116
-
Again the development happens on the host. Suppoose that we have a simple application program in `a.py`, we can test and run it using the production image:
116
+
Again the development happens on the host. Suppose that we have a simple application program in `a.py`, we can test and run it using the production image:
117
117
118
118
```bash
119
119
docker run -it -v $PWD:/work paddle /work/a.py
120
120
```
121
121
122
122
But this works only if all dependencies of `a.py` are in the production image. If this is not the case, we need to build a new Docker image from the production image and with more dependencies installs.
123
123
124
-
### Build and Run PaddlePaddle Appications
124
+
### Build and Run PaddlePaddle Applications
125
125
126
126
We need a Dockerfile in https://github.com/paddlepaddle/book that builds Docker image `paddlepaddle/book:<version>`, basing on the PaddlePaddle production image:
127
127
@@ -143,7 +143,7 @@ docker build -t book .
143
143
144
144
### Build and Run Distributed Applications
145
145
146
-
In our [API design doc](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/api.md#distributed-training), we proposed an API that starts a distributed training job on a cluster. This API need to build a PaddlePaddle application into a Docekr image as above, and calls kubectl to run it on the cluster. This API might need to generate a Dockerfile look like above and call `docker build`.
146
+
In our [API design doc](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/api.md#distributed-training), we proposed an API that starts a distributed training job on a cluster. This API need to build a PaddlePaddle application into a Docker image as above and calls kubectl to run it on the cluster. This API might need to generate a Dockerfile look like above and call `docker build`.
147
147
148
148
Of course, we can manually build an application image and launch the job using the kubectl tool:
0 commit comments