Skip to content

Commit bf59a2a

Browse files
RanjitCSRanjitCS
committed
Move docs from CBCI page to github
1 parent 9cda71b commit bf59a2a

File tree

3 files changed

+197
-1
lines changed

3 files changed

+197
-1
lines changed

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ Jenkins plugin which allows building, testing, and using Docker images from Jenk
1010
Summary
1111
---
1212

13-
A full description is available in the plugin’s [documentation](https://go.cloudbees.com/docs/plugins/docker-workflow/).
13+
A full description is available in the plugin’s [documentation](docs/docker-workflow.md/).
1414

1515
Demo
1616
---

‎docs/docker-workflow.md‎

Lines changed: 196 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,196 @@
1+
# Docker Pipeline Plugin
2+
3+
## Introduction
4+
5+
Many organizations are using [Docker](https://www.docker.com) to unify their build and test environments across machines and provide an efficient way to deploy applications into production. This plugin offers a convenient domain-specific language (DSL) for performing some of the most commonly needed Docker operations in a continuous-deployment pipeline from a Pipeline script.
6+
7+
The entry point for all of this plugin's functionality is a `docker` global variable, available without import to any flow script when the plugin is enabled. To get detailed information on available methods, open the _Pipeline Syntax_ available on any Pipeline Job page:
8+
9+
**Global variable reference**
10+
![Global variable reference](dsl-help.png)
11+
12+
## Run Build Steps Inside Containers
13+
14+
It is commonplace for Jenkins projects to require a specific toolset or libraries to be available during a build. If many projects in a Jenkins installation have the same requirements, and there are few agents, it is not hard to just preconfigure those agents accordingly. In other cases it is feasible to keep such files in project source control. Finally, for some tools--especially those with a self-contained, platform-independent download, like Maven--it is possible to use the Jenkins tool installer system with the Pipeline `tool` step to retrieve tools on demand. However, many cases remain where these techniques are not practical.
15+
16+
For builds which can run on Linux, Docker provides an ideal solution to this problem. Each project need merely select an image containing all the tools and libraries it would need. (This might be a publicly available image like [maven](https://registry.hub.docker.com/_/maven/), or it might have been built by this or another Jenkins project.) Developers can also run build steps locally using an environment identical to that used by the Jenkins project.
17+
18+
There are two ways to run Jenkins build steps in such an image. One is to include a Java runtime and Jenkins agent JAR file inside the image, and add a _Docker_ cloud using the Docker plugin. Then the entire agent runs wholly inside the image, so any builds tied to that cloud label can assume a particular environment.
19+
20+
For cases where you do not want to "pollute" the image with Java and the Jenkins agent, or just want a simpler and more flexible setup, this plugin provides a way to run build steps inside an arbitrary image. In the simplest case, just select an `image` and run your whole build `inside` it:
21+
22+
```groovy
23+
docker.image('maven:3.3.3-jdk-8').inside {
24+
git '…your-sources…'
25+
sh 'mvn -B clean install'
26+
}
27+
```
28+
29+
The above is a complete Pipeline script. `inside` will:
30+
31+
1. Automatically grab an agent and a workspace (no extra `node` block is required).
32+
2. Pull the requested image to the Docker server (if not already cached).
33+
3. Start a container running that image.
34+
4. Mount the Jenkins workspace as a "volume" inside the container, using the same file path.
35+
5. Run your build steps. External processes like `sh` will be wrapped in `docker exec` so they are run inside the container. Other steps (such as test reporting) run unmodified: they can still access workspace files created by build steps.
36+
6. At the end of the block, stop the container and discard any storage it consumed.
37+
7. Record the fact that the build used the specified image. This unlocks features in other Jenkins plugins: you can track all projects using an image, or configure this project to be triggered automatically when an updated image is pushed to the Docker registry. If you use the Docker Traceability plugin, you will be also able to see a history of the image deployments on Docker servers.
38+
39+
> **TIP:**
40+
> If you are running a tool like Maven which has a large download cache, running each build inside its own image will mean that a large quantity of data is downloaded from the network on every build, which is usually undesirable. The easiest way to avoid this is to redirect the cache to the agent workspace, so that if you run another build on the same agent, it will run much more quickly. In the case of Maven:
41+
>
42+
> ```groovy
43+
> docker.image('maven:3.3.3-jdk-8').inside {
44+
> git '…your-sources…'
45+
> writeFile file: 'settings.xml', text: "<settings><localRepository>${pwd()}/.m2repo</localRepository></settings>"
46+
> sh 'mvn -B -s settings.xml clean install'
47+
> }
48+
> ```
49+
>
50+
> (If you wanted to use a cache location elsewhere on the agent, you would need to pass an extra `--volume` option to `inside` so that the container could see that path.)
51+
>
52+
> Another solution is to pass an argument to `inside` to mount a sharable volume, such as `-v m2repo:/m2repo`, and use that path as the `localRepository`. Just beware that the default local repository management in Maven is not thread-safe for concurrent builds, and `install:install` could pollute the local repository across builds or even across jobs. The safest solution is to use a nearby repository mirror as a cache.
53+
54+
> **NOTE:**
55+
> For `inside` to work, the Docker server and the Jenkins agent must use the same filesystem, so that the workspace can be mounted. The easiest way to ensure this is for the Docker server to be running on localhost (the same computer as the agent). Currently, neither the Jenkins plugin or the Docker CLI automatically detect that the server is running remotely; typical symptoms are negative exit codes or errors from nested `sh` commands, such as:
56+
>
57+
> ```bash
58+
> cannot create /…@tmp/durable-…/pid: Directory nonexistent
59+
> ```
60+
>
61+
> When Jenkins can detect that the agent is itself running inside a Docker container, it automatically passes the `--volumes-from` argument to the `inside` container, ensuring that it can share a workspace with the agent.
62+
63+
## Customize Agent Allocation
64+
65+
All DSL functions which run some Docker command automatically acquire an agent (executor) and a workspace if necessary. For more complex scripts which perform several commands using the DSL, you will typically want to run a block, or the whole script, on the _same_ agent and workspace. In that case just wrap the block in `node`, selecting a label if desired:
66+
67+
```groovy
68+
node('linux') {
69+
def maven = docker.image('maven:latest')
70+
maven.pull() // make sure we have the latest available from Docker Hub
71+
maven.inside {
72+
// …as above
73+
}
74+
}
75+
```
76+
77+
Here we ensure that the same agent runs both `pull` and `inside`, so the local image cache update by the first step is seen by the second.
78+
79+
## Build and Publish Images
80+
81+
If your build needs to create a Docker image, use the `build` method, which takes an image name with an optional tag and creates it from a `Dockerfile`. This also returns a handle to the result image, so you can work with it further:
82+
83+
```groovy
84+
node {
85+
git '…' // checks out Dockerfile & Makefile
86+
def myEnv = docker.build 'my-environment:snapshot'
87+
myEnv.inside {
88+
sh 'make test'
89+
}
90+
}
91+
```
92+
93+
Here the `build` method takes a `Dockerfile` in your source tree specifying a build environment (for example `RUN apt-get install -y libapr1-dev`). Then a `Makefile` in the same source tree describes how to build your actual project in that environment.
94+
95+
If you want to publish a newly created image to Docker Hub (or your own registry—discussed below), use `push`:
96+
97+
```groovy
98+
node {
99+
git '…' // checks out Dockerfile and some project sources
100+
def newApp = docker.build "mycorp/myapp:${env.BUILD_TAG}"
101+
newApp.push()
102+
}
103+
```
104+
105+
Here we are giving the image a tag which identifies the Jenkins project and build number that created it. (See the documentation for the `env` global variable.) The image is pushed under this tag name to the registry.
106+
107+
To push an image into a staging or production environment, a common style is to update a predefined tag such as `latest` in the registry. In this case just specify the tag name:
108+
109+
```groovy
110+
node {
111+
stage 'Building image'
112+
git '…'
113+
def newApp = docker.build "mycorp/myapp:${env.BUILD_TAG}"
114+
newApp.push() // record this snapshot (optional)
115+
stage 'Test image'
116+
// run some tests on it (see below), then if everything looks good:
117+
stage 'Approve image'
118+
newApp.push 'latest'
119+
}
120+
```
121+
122+
The `build` method records information in Jenkins tied to this project build: what image was built, and what image that was derived from (the `FROM` instruction at the top of your `Dockerfile`). Other plugins can then identify the build which created an image known to have been used by a downstream build, or deployed to a particular environment. You can also have this project be triggered when an update is pushed to the ancestor image (`FROM`) in a registry.
123+
124+
## Run and Test Containers
125+
126+
To run an image you built, or pulled from a registry, you can use the `run` method. This returns a handle to the running container. More safely, you can use the `withRun` method, which automatically stops the container at the end of a block:
127+
128+
```groovy
129+
node {
130+
git '…'
131+
docker.image('mysql').withRun {c ->
132+
sh './test-with-local-db'
133+
}
134+
}
135+
```
136+
137+
The above simply starts a container running a test MySQL database and runs a regular build while that container is running. Unlike `inside`, shell steps inside the block are **not** run inside the container, but they could connect to it using a local TCP port for example.
138+
139+
You can also access the `id` of the running container, which is passed as an argument to the block, in case you need to do anything further with it:
140+
141+
```groovy
142+
// …as above, but also dump logs before we finish:
143+
sh "docker logs ${c.id}"
144+
```
145+
146+
Like `inside`, `run` and `withRun` record the fact that the build used the specified image.
147+
148+
## Specify a Custom Registry and Server
149+
150+
So far we have assumed that you are using the public Docker Hub as the image registry, and connecting to a Docker server in the default location (typically a daemon running locally on a Linux agent). Either or both of these settings can be easily customized.
151+
152+
To select a custom registry, wrap build steps which need it in the `withRegistry` method on `docker` (inside `node` if you want to specify an agent explicitly). You should pass in a registry URL. If the registry requires authentication, you can add the ID of username/password credentials. (_Credentials_ link in the Jenkins index page or in a folder; when creating the credentials, use the _Advanced_ section to specify a memorable ID for use in your pipelines.)
153+
154+
```groovy
155+
docker.withRegistry('https://docker.mycorp.com/', 'docker-login') {
156+
git '…'
157+
docker.build('myapp').push('latest')
158+
}
159+
```
160+
161+
The above builds an image from a `Dockerfile`, and then publishes it (under the `latest` tag) to a password-protected registry. There is no need to preconfigure authentication on the agent.
162+
163+
To select a non-default Docker server, such as for Docker Swarm, use the `withServer` method. You pass in a URI, and optionally the ID of _Docker Server Certificate Authentication_ credentials (which encode a client key and client/server certificates to support TLS).
164+
165+
```groovy
166+
docker.withServer('tcp://swarm.mycorp.com:2376', 'swarm-certs') {
167+
docker.image('httpd').withRun('-p 8080:80') {c ->
168+
sh "curl -i http://${hostIp(c)}:8080/"
169+
}
170+
}
171+
def hostIp(container) {
172+
sh "docker inspect -f {{.Node.Ip}} ${container.id} > hostIp"
173+
readFile('hostIp').trim()
174+
}
175+
```
176+
177+
Note that you cannot use `inside` or `build` with a Swarm server, and some versions of Swarm do not support interacting with a custom registry either.
178+
179+
## Advanced Usage
180+
181+
If your script needs to run other Docker client commands or options not covered by the DSL, use a `sh` step. You can still take advantage of some DSL methods like `imageName` to prepend a registry ID:
182+
183+
```groovy
184+
docker.withRegistry('https://docker.mycorp.com/') {
185+
def myImg = docker.image('myImg')
186+
// or docker.build, etc.
187+
sh "docker pull --all-tags ${myImg.imageName()}"
188+
// runs: docker pull --all-tags docker.mycorp.com/myImg
189+
}
190+
```
191+
192+
and you can be assured that the environment variables and files needed to connect to any custom registry and/or server will be prepared.
193+
194+
## Demonstrations
195+
196+
A Docker image is available for you to run which demonstrates a complete flow script which builds an application as an image, tests it from another container, and publishes the result to a private registry. Refer to the [Instructions on running this demonstration](https://github.com/jenkinsci/docker-workflow-plugin/tree/docker-workflow-1.12/demo) for details.

‎docs/dsl-help.png‎

353 KB
Loading

0 commit comments

Comments
 (0)