Skip to content

Commit 9f1daa6

Browse files
slonopotamusolljanat
authored andcommitted
Editorial pass over README
* Fix release badge * Fix typos * Enable syntax highlighting * etc
1 parent 9076dfe commit 9f1daa6

File tree

1 file changed

+68
-54
lines changed

1 file changed

+68
-54
lines changed

README.md

Lines changed: 68 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,25 @@
11
# nomad-driver-containerd
2+
23
[![CI Actions Status](https://github.com/Roblox/nomad-driver-containerd/workflows/CI/badge.svg)](https://github.com/Roblox/nomad-driver-containerd/actions)
34
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/Roblox/nomad-driver-containerd/blob/master/LICENSE)
4-
[![Release](https://img.shields.io/badge/version-0.9.3-blue)](https://github.com/Roblox/nomad-driver-containerd/releases/tag/v0.9.3)
5+
[![Release](https://img.shields.io/github/release/Roblox/nomad-driver-containerd.svg)](https://github.com/Roblox/nomad-driver-containerd/releases/latest)
56
[![Docs](https://img.shields.io/badge/docs-website-green.svg)](https://www.nomadproject.io/docs/drivers/external/containerd)
67

78
<img src="images/nomad.png" width="232" height="100" />&nbsp;<img src="images/docker.png" width="116" height="100" />&nbsp;<img src="images/containerd.png" width="285" height="100" />
89

910
We are actively looking for contributors and maintainers for this project.
1011
If you have experience in container internals e.g. cgroups, namespaces, or have contributed to any open source projects
11-
around containers e.g. [`docker`](https://github.com/moby/moby), [`containerd`](https://github.com/containerd/containerd), [`nerdctl`](https://github.com/containerd/nerdctl), [`podman`](https://github.com/containers/podman) etc or build tooling which involves dealing with
12+
around containers e.g. [`docker`](https://github.com/moby/moby), [`containerd`](https://github.com/containerd/containerd), [`nerdctl`](https://github.com/containerd/nerdctl), [`podman`](https://github.com/containers/podman) etc. or build tooling which involves dealing with
1213
container internals, and are interested in contributing to this project, I would love to talk to you!
1314
Golang experience is preferred but not required.
1415

1516
Please reach out to me [`@_shishir_m`](https://twitter.com/_shishir_m) or open an issue in this repository with your contact details, if you are interested in contributing to this project.
1617

1718
## Overview
19+
1820
Nomad task driver for launching containers using containerd.
1921

20-
**Containerd** [`(containerd.io)`](https://containerd.io) is a lightweight container daemon for
21-
running and managing container lifecycle.<br/>
22+
**Containerd** [`(containerd.io)`](https://containerd.io) is a lightweight container daemon for running and managing container lifecycle.<br/>
2223
Docker daemon also uses containerd.
2324

2425
```
@@ -29,6 +30,7 @@ dockerd (docker daemon) --> containerd --> containerd-shim --> runc
2930
Docker daemon is not required on the host system.
3031

3132
## nomad-driver-containerd architecture
33+
3234
<img src="images/nomad_driver_containerd.png" width="850" height="475" />
3335

3436
## Requirements
@@ -41,51 +43,52 @@ Docker daemon is not required on the host system.
4143

4244
## Building nomad-driver-containerd
4345

44-
Make sure your **$GOPATH** is setup correctly.
45-
```
46-
$ mkdir -p $GOPATH/src/github.com/Roblox
47-
$ cd $GOPATH/src/github.com/Roblox
46+
Make sure your **$GOPATH** is set up correctly.
47+
48+
```shell
4849
$ git clone [email protected]:Roblox/nomad-driver-containerd.git
4950
$ cd nomad-driver-containerd
50-
$ make build (This will build your containerd-driver binary)
51+
$ make build # This will build your containerd-driver binary
5152
```
5253

5354
If you want to compile for `arm64`, you can run:
5455

55-
```
56-
make -f Makefile.arm64
56+
```shell
57+
$ make -f Makefile.arm64
5758
```
5859

5960
## Screencast
61+
6062
[![asciicast](https://asciinema.org/a/348173.svg)](https://asciinema.org/a/348173)
6163

6264
## Wanna try it out!?
6365

64-
```
66+
```shell
6567
$ vagrant up
6668
```
6769
or `vagrant provision` if the vagrant VM is already running.
6870

69-
Once setup (`vagrant up` OR `vagrant provision`) is complete and the nomad server is up and running, you can check the registered task drivers (which will also show `containerd-driver`) using:
70-
```
71-
$ nomad node status (Note down the <node_id>)
71+
Once setup is complete and the nomad server is up and running, you can check the registered task drivers (which will also show `containerd-driver`) using:
72+
```shell
73+
$ nomad node status # Note down node ID
7274
$ nomad node status <node_id> | grep containerd-driver
7375
```
7476

7577
**NOTE:** [`setup.sh`](vagrant/setup.sh) is part of the vagrant setup and should not be executed directly.
7678

77-
## Run Example jobs.
79+
## Run example jobs
7880

7981
There are few example jobs in the [`example`](https://github.com/Roblox/nomad-driver-containerd/tree/master/example) directory.
8082

83+
```shell
84+
$ nomad job run <job_name>.nomad
8185
```
82-
$ nomad job run <job_name.nomad>
83-
```
84-
will launch the job.<br/>
86+
will launch the job.
8587

86-
More detailed instructions are in the [`example README.md`](https://github.com/Roblox/nomad-driver-containerd/tree/master/example)
88+
More detailed instructions are available in the [`example README.md`](https://github.com/Roblox/nomad-driver-containerd/tree/master/example).
8789

88-
To interact with `images` and `containers` directly, you can use [`nerdctl`](https://github.com/containerd/nerdctl) which is a docker compatible CLI for `containerd`. `nerdctl` is already installed in the vagrant VM at `/usr/local/bin`.
90+
To interact with `images` and `containers` directly, you can use [`nerdctl`](https://github.com/containerd/nerdctl) which is a docker compatible CLI for `containerd`.
91+
`nerdctl` is already installed in the vagrant VM at `/usr/local/bin`.
8992

9093
## Supported options
9194

@@ -134,11 +137,12 @@ To interact with `images` and `containers` directly, you can use [`nerdctl`](htt
134137
&emsp;&emsp;&emsp;- **type** (string) (Optional): Supported values are `volume`, `bind` or `tmpfs`. **Default:** volume.<br/>
135138
&emsp;&emsp;&emsp;- **target** (string) (Required): Target path in the container.<br/>
136139
&emsp;&emsp;&emsp;- **source** (string) (Optional): Source path on the host.<br/>
137-
&emsp;&emsp;&emsp;- **options** ([]string) (Optional): fstab style [`mount options`](https://github.com/containerd/containerd/blob/master/mount/mount_linux.go#L187-L211). **NOTE**: For bind mounts, atleast `rbind` and `ro` are required.<br/>
140+
&emsp;&emsp;&emsp;- **options** ([]string) (Optional): fstab style [`mount options`](https://github.com/containerd/containerd/blob/master/mount/mount_linux.go#L187-L211). **NOTE**: For bind mounts, at least `rbind` and `ro` are required.<br/>
138141
&emsp;&emsp;\}
139142

140143
**Bind mount example**
141-
```
144+
145+
```hcl
142146
mounts = [
143147
{
144148
type = "bind"
@@ -149,18 +153,18 @@ mounts = [
149153
]
150154
```
151155

152-
In additon to the `mounts` option in `Task Config`, you can also mount your volumes into the container using nomad [`volume_mount stanza`](https://www.nomadproject.io/docs/job-specification/volume_mount)
156+
In addition to the `mounts` option in `Task Config`, you can also mount your volumes into the container using nomad [`volume_mount stanza`](https://www.nomadproject.io/docs/job-specification/volume_mount)
153157

154158
See [`example job`](https://github.com/Roblox/nomad-driver-containerd/blob/master/example/volume_mount.nomad) for `volume_mount`.
155159

156160
**Custom seccomp profile example**
157161

158-
The default `docker` seccomp profile found [`here`](https://github.com/moby/moby/blob/master/profiles/seccomp/default.json)
159-
can be downloaded, and modified (by removing/adding syscalls) to create a custom seccomp profile.<br/>
162+
The default `docker` seccomp profile found [`here`](https://github.com/moby/moby/blob/master/profiles/seccomp/default.json) can be downloaded and modified (by adding/removing syscalls) to create a custom seccomp profile.
160163
The custom seccomp profile can then be saved under `/opt/seccomp/seccomp.json` on the Nomad client nodes.
161164

162165
A nomad job can be launched using this custom seccomp profile.
163-
```
166+
167+
```hcl
164168
config {
165169
seccomp = true
166170
seccomp_profile = "/opt/seccomp/seccomp.json"
@@ -169,7 +173,7 @@ config {
169173

170174
**Sysctl example**
171175

172-
```
176+
```hcl
173177
config {
174178
sysctl = {
175179
"net.core.somaxconn" = "16384"
@@ -185,9 +189,10 @@ an image from a private repository in docker hub.<br/>
185189
`auth` stanza can be set either in `Driver Config` or `Task Config` or both.<br/>
186190
If set at both places, `Task Config` auth will take precedence over `Driver Config` auth.
187191

188-
**NOTE**: In the below example, `user` and `pass` are just placeholder values which need to be replaced by actual `username` and `password`, when specifying the credentials. Below `auth` stanza can be used for both `Driver Config` and `Task Config`.
192+
**NOTE**: In the example below, `user` and `pass` are just placeholder values which need to be replaced by actual `username` and `password`, when specifying the credentials.
193+
`auth` stanza can be used for both `Driver Config` and `Task Config`.
189194

190-
```
195+
```hcl
191196
auth {
192197
username = "user"
193198
password = "pass"
@@ -200,34 +205,39 @@ auth {
200205

201206
**NOTE:** `host` and `bridge` are mutually exclusive options, and only one of them should be used at a time.
202207

203-
1. **Host** network can be enabled by setting `host_network` to `true` in task config
204-
of the job spec (see under [`Supported options`](https://github.com/Roblox/nomad-driver-containerd#supported-options)).
208+
1. **Host** network can be enabled by setting `host_network` to `true` in task config of the job spec (see under [`Supported options`](#supported-options)).
205209

206210
2. **Bridge** network can be enabled by setting the `network` stanza in the task group section of the job spec.
207211

208-
```
212+
```hcl
209213
network {
210214
mode = "bridge"
211215
}
212216
```
217+
213218
You need to install CNI plugins on Nomad client nodes under `/opt/cni/bin` before you can use `bridge` networks.
214219

215-
**Instructions for installing CNI plugins.**<br/>
216-
```
217-
$ curl -L -o cni-plugins.tgz https://github.com/containernetworking/plugins/releases/download/v0.8.6/cni-plugins-linux-amd64-v0.8.6.tgz
218-
$ sudo mkdir -p /opt/cni/bin
219-
$ sudo tar -C /opt/cni/bin -xzf cni-plugins.tgz
220-
```
221-
Also, ensure your Linux operating system distribution has been configured to allow container traffic through the bridge network to be routed via iptables. These tunables can be set as follows:
220+
**Installing CNI plugins**
222221

222+
[//]: # (TODO: Replace this with containernetworking-plugins Ubuntu package)
223+
224+
```shell
225+
$ curl -L -o cni-plugins.tgz https://github.com/containernetworking/plugins/releases/download/v0.8.6/cni-plugins-linux-amd64-v0.8.6.tgz
226+
$ sudo mkdir -p /opt/cni/bin
227+
$ sudo tar -C /opt/cni/bin -xzf cni-plugins.tgz
223228
```
229+
230+
Also, ensure your Linux operating system distribution has been configured to allow container traffic through the bridge network to be routed via iptables.
231+
These tunables can be set as follows:
232+
233+
```shell
224234
$ echo 1 > /proc/sys/net/bridge/bridge-nf-call-arptables
225235
$ echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables
226236
$ echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
227237
```
228238
To preserve these settings on startup of a nomad client node, add a file including the following to `/etc/sysctl.d/` or remove the file your Linux distribution puts in that directory.
229239

230-
```
240+
```toml
231241
net.bridge.bridge-nf-call-arptables = 1
232242
net.bridge.bridge-nf-call-ip6tables = 1
233243
net.bridge.bridge-nf-call-iptables = 1
@@ -237,10 +247,10 @@ net.bridge.bridge-nf-call-iptables = 1
237247

238248
nomad supports both **static** and **dynamic** port mapping.
239249

240-
1. **Static ports**
250+
### Static ports
241251

242252
Static port mapping can be added in the `network` stanza.
243-
```
253+
```hcl
244254
network {
245255
mode = "bridge"
246256
port "lb" {
@@ -252,10 +262,10 @@ network {
252262
Here, `host` port `8889` is mapped to `container` port `8889`.<br/>
253263
**NOTE**: static ports are usually not recommended, except for `system` or specialized jobs like load balancers.
254264

255-
2. **Dynamic ports**
265+
### Dynamic ports
256266

257267
Dynamic port mapping is also enabled in the `network` stanza.
258-
```
268+
```hcl
259269
network {
260270
mode = "bridge"
261271
port "http" {
@@ -265,11 +275,13 @@ network {
265275
```
266276
Here, nomad will allocate a dynamic port on the `host` and that port will be mapped to `8080` in the container.
267277

268-
You can also read more about `network stanza` in the [`nomad official documentation`](https://www.nomadproject.io/docs/job-specification/network)
278+
You can also read more about `network stanza` in the [`nomad official documentation`](https://www.nomadproject.io/docs/job-specification/network).
269279

270280
## Service discovery
271281

272-
Nomad schedules workloads of various types across a cluster of generic hosts. Because of this, placement is not known in advance and you will need to use service discovery to connect tasks to other services deployed across your cluster. Nomad integrates with Consul to provide service discovery and monitoring.
282+
Nomad schedules workloads of various types across a cluster of generic hosts.
283+
Because of this, placement is not known in advance, and you will need to use service discovery to connect tasks to other services deployed across your cluster.
284+
Nomad integrates with Consul to provide service discovery and monitoring.
273285

274286
A [`service`](https://www.nomadproject.io/docs/job-specification/service) stanza can be added to your job spec, to enable service discovery.
275287

@@ -279,34 +291,36 @@ The service stanza instructs Nomad to register a service with Consul.
279291

280292
If you are running the tests locally, use the [`vagrant VM`](Vagrantfile) provided in the repository.
281293

282-
```
294+
```shell
283295
$ vagrant up
284296
$ vagrant ssh containerd-linux
285297
$ sudo make test
286298
```
287299
**NOTE**: These are destructive tests and can leave the system in a changed state.<br/>
288300
It is highly recommended to run these tests either as part of a CI/CD system e.g. circleci or on
289-
a immutable infrastructure e.g vagrant VMs.
301+
an immutable infrastructure e.g. vagrant VMs.
290302

291303
You can also run an individual test by specifying the test name. e.g.
292304

293-
```
305+
```shell
294306
$ cd tests
295307
$ sudo ./run_tests.sh 001-test-redis.sh
296308
```
297309

298310
## Cleanup
299-
```
300-
make clean
311+
312+
```shell
313+
$ make clean
301314
```
302315
This will delete your binary: `containerd-driver`
303316

304-
```
305-
vagrant destroy
317+
```shell
318+
$ vagrant destroy
306319
```
307320
This will destroy your vagrant VM.
308321

309322
## Currently supported environments
323+
310324
Ubuntu (>= 16.04)
311325

312326
## License

0 commit comments

Comments
 (0)