Skip to content

Commit 958246c

Browse files
slonopotamusolljanat
authored andcommitted
Editorial pass over README
* Fix release badge * Fix typos * Enable syntax highlighting * etc
1 parent 6b59b93 commit 958246c

File tree

1 file changed

+68
-54
lines changed

1 file changed

+68
-54
lines changed

README.md

Lines changed: 68 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,25 @@
11
# nomad-driver-containerd
2+
23
[![CI Actions Status](https://github.com/Roblox/nomad-driver-containerd/workflows/CI/badge.svg)](https://github.com/Roblox/nomad-driver-containerd/actions)
34
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/Roblox/nomad-driver-containerd/blob/master/LICENSE)
4-
[![Release](https://img.shields.io/badge/version-0.9.3-blue)](https://github.com/Roblox/nomad-driver-containerd/releases/tag/v0.9.3)
5+
[![Release](https://img.shields.io/github/release/Roblox/nomad-driver-containerd.svg)](https://github.com/Roblox/nomad-driver-containerd/releases/latest)
56
[![Docs](https://img.shields.io/badge/docs-website-green.svg)](https://www.nomadproject.io/docs/drivers/external/containerd)
67

78
<img src="images/nomad.png" width="232" height="100" />&nbsp;<img src="images/docker.png" width="116" height="100" />&nbsp;<img src="images/containerd.png" width="285" height="100" />
89

910
We are actively looking for contributors and maintainers for this project.
1011
If you have experience in container internals e.g. cgroups, namespaces, or have contributed to any open source projects
11-
around containers e.g. [`docker`](https://github.com/moby/moby), [`containerd`](https://github.com/containerd/containerd), [`nerdctl`](https://github.com/containerd/nerdctl), [`podman`](https://github.com/containers/podman) etc or build tooling which involves dealing with
12+
around containers e.g. [`docker`](https://github.com/moby/moby), [`containerd`](https://github.com/containerd/containerd), [`nerdctl`](https://github.com/containerd/nerdctl), [`podman`](https://github.com/containers/podman) etc. or build tooling which involves dealing with
1213
container internals, and are interested in contributing to this project, I would love to talk to you!
1314
Golang experience is preferred but not required.
1415

1516
Please reach out to me [`@_shishir_m`](https://twitter.com/_shishir_m) or open an issue in this repository with your contact details, if you are interested in contributing to this project.
1617

1718
## Overview
19+
1820
Nomad task driver for launching containers using containerd.
1921

20-
**Containerd** [`(containerd.io)`](https://containerd.io) is a lightweight container daemon for
21-
running and managing container lifecycle.<br/>
22+
**Containerd** [`(containerd.io)`](https://containerd.io) is a lightweight container daemon for running and managing container lifecycle.<br/>
2223
Docker daemon also uses containerd.
2324

2425
```
@@ -29,6 +30,7 @@ dockerd (docker daemon) --> containerd --> containerd-shim --> runc
2930
Docker daemon is not required on the host system.
3031

3132
## nomad-driver-containerd architecture
33+
3234
<img src="images/nomad_driver_containerd.png" width="850" height="475" />
3335

3436
## Requirements
@@ -41,51 +43,52 @@ Docker daemon is not required on the host system.
4143

4244
## Building nomad-driver-containerd
4345

44-
Make sure your **$GOPATH** is setup correctly.
45-
```
46-
$ mkdir -p $GOPATH/src/github.com/Roblox
47-
$ cd $GOPATH/src/github.com/Roblox
46+
Make sure your **$GOPATH** is set up correctly.
47+
48+
```shell
4849
$ git clone [email protected]:Roblox/nomad-driver-containerd.git
4950
$ cd nomad-driver-containerd
50-
$ make build (This will build your containerd-driver binary)
51+
$ make build # This will build your containerd-driver binary
5152
```
5253

5354
If you want to compile for `arm64`, you can run:
5455

55-
```
56-
make -f Makefile.arm64
56+
```shell
57+
$ make -f Makefile.arm64
5758
```
5859

5960
## Screencast
61+
6062
[![asciicast](https://asciinema.org/a/348173.svg)](https://asciinema.org/a/348173)
6163

6264
## Wanna try it out!?
6365

64-
```
66+
```shell
6567
$ vagrant up
6668
```
6769
or `vagrant provision` if the vagrant VM is already running.
6870

69-
Once setup (`vagrant up` OR `vagrant provision`) is complete and the nomad server is up and running, you can check the registered task drivers (which will also show `containerd-driver`) using:
70-
```
71-
$ nomad node status (Note down the <node_id>)
71+
Once setup is complete and the nomad server is up and running, you can check the registered task drivers (which will also show `containerd-driver`) using:
72+
```shell
73+
$ nomad node status # Note down node ID
7274
$ nomad node status <node_id> | grep containerd-driver
7375
```
7476

7577
**NOTE:** [`setup.sh`](vagrant/setup.sh) is part of the vagrant setup and should not be executed directly.
7678

77-
## Run Example jobs.
79+
## Run example jobs
7880

7981
There are few example jobs in the [`example`](https://github.com/Roblox/nomad-driver-containerd/tree/master/example) directory.
8082

83+
```shell
84+
$ nomad job run <job_name>.nomad
8185
```
82-
$ nomad job run <job_name.nomad>
83-
```
84-
will launch the job.<br/>
86+
will launch the job.
8587

86-
More detailed instructions are in the [`example README.md`](https://github.com/Roblox/nomad-driver-containerd/tree/master/example)
88+
More detailed instructions are available in the [`example README.md`](https://github.com/Roblox/nomad-driver-containerd/tree/master/example).
8789

88-
To interact with `images` and `containers` directly, you can use [`nerdctl`](https://github.com/containerd/nerdctl) which is a docker compatible CLI for `containerd`. `nerdctl` is already installed in the vagrant VM at `/usr/local/bin`.
90+
To interact with `images` and `containers` directly, you can use [`nerdctl`](https://github.com/containerd/nerdctl) which is a docker compatible CLI for `containerd`.
91+
`nerdctl` is already installed in the vagrant VM at `/usr/local/bin`.
8992

9093
## Supported options
9194

@@ -135,11 +138,12 @@ To interact with `images` and `containers` directly, you can use [`nerdctl`](htt
135138
&emsp;&emsp;&emsp;- **type** (string) (Optional): Supported values are `volume`, `bind` or `tmpfs`. **Default:** volume.<br/>
136139
&emsp;&emsp;&emsp;- **target** (string) (Required): Target path in the container.<br/>
137140
&emsp;&emsp;&emsp;- **source** (string) (Optional): Source path on the host.<br/>
138-
&emsp;&emsp;&emsp;- **options** ([]string) (Optional): fstab style [`mount options`](https://github.com/containerd/containerd/blob/master/mount/mount_linux.go#L187-L211). **NOTE**: For bind mounts, atleast `rbind` and `ro` are required.<br/>
141+
&emsp;&emsp;&emsp;- **options** ([]string) (Optional): fstab style [`mount options`](https://github.com/containerd/containerd/blob/master/mount/mount_linux.go#L187-L211). **NOTE**: For bind mounts, at least `rbind` and `ro` are required.<br/>
139142
&emsp;&emsp;\}
140143

141144
**Bind mount example**
142-
```
145+
146+
```hcl
143147
mounts = [
144148
{
145149
type = "bind"
@@ -150,18 +154,18 @@ mounts = [
150154
]
151155
```
152156

153-
In additon to the `mounts` option in `Task Config`, you can also mount your volumes into the container using nomad [`volume_mount stanza`](https://www.nomadproject.io/docs/job-specification/volume_mount)
157+
In addition to the `mounts` option in `Task Config`, you can also mount your volumes into the container using nomad [`volume_mount stanza`](https://www.nomadproject.io/docs/job-specification/volume_mount)
154158

155159
See [`example job`](https://github.com/Roblox/nomad-driver-containerd/blob/master/example/volume_mount.nomad) for `volume_mount`.
156160

157161
**Custom seccomp profile example**
158162

159-
The default `docker` seccomp profile found [`here`](https://github.com/moby/moby/blob/master/profiles/seccomp/default.json)
160-
can be downloaded, and modified (by removing/adding syscalls) to create a custom seccomp profile.<br/>
163+
The default `docker` seccomp profile found [`here`](https://github.com/moby/moby/blob/master/profiles/seccomp/default.json) can be downloaded and modified (by adding/removing syscalls) to create a custom seccomp profile.
161164
The custom seccomp profile can then be saved under `/opt/seccomp/seccomp.json` on the Nomad client nodes.
162165

163166
A nomad job can be launched using this custom seccomp profile.
164-
```
167+
168+
```hcl
165169
config {
166170
seccomp = true
167171
seccomp_profile = "/opt/seccomp/seccomp.json"
@@ -170,7 +174,7 @@ config {
170174

171175
**Sysctl example**
172176

173-
```
177+
```hcl
174178
config {
175179
sysctl = {
176180
"net.core.somaxconn" = "16384"
@@ -186,9 +190,10 @@ an image from a private repository in docker hub.<br/>
186190
`auth` stanza can be set either in `Driver Config` or `Task Config` or both.<br/>
187191
If set at both places, `Task Config` auth will take precedence over `Driver Config` auth.
188192

189-
**NOTE**: In the below example, `user` and `pass` are just placeholder values which need to be replaced by actual `username` and `password`, when specifying the credentials. Below `auth` stanza can be used for both `Driver Config` and `Task Config`.
193+
**NOTE**: In the example below, `user` and `pass` are just placeholder values which need to be replaced by actual `username` and `password`, when specifying the credentials.
194+
`auth` stanza can be used for both `Driver Config` and `Task Config`.
190195

191-
```
196+
```hcl
192197
auth {
193198
username = "user"
194199
password = "pass"
@@ -201,34 +206,39 @@ auth {
201206

202207
**NOTE:** `host` and `bridge` are mutually exclusive options, and only one of them should be used at a time.
203208

204-
1. **Host** network can be enabled by setting `host_network` to `true` in task config
205-
of the job spec (see under [`Supported options`](https://github.com/Roblox/nomad-driver-containerd#supported-options)).
209+
1. **Host** network can be enabled by setting `host_network` to `true` in task config of the job spec (see under [`Supported options`](#supported-options)).
206210

207211
2. **Bridge** network can be enabled by setting the `network` stanza in the task group section of the job spec.
208212

209-
```
213+
```hcl
210214
network {
211215
mode = "bridge"
212216
}
213217
```
218+
214219
You need to install CNI plugins on Nomad client nodes under `/opt/cni/bin` before you can use `bridge` networks.
215220

216-
**Instructions for installing CNI plugins.**<br/>
217-
```
218-
$ curl -L -o cni-plugins.tgz https://github.com/containernetworking/plugins/releases/download/v0.8.6/cni-plugins-linux-amd64-v0.8.6.tgz
219-
$ sudo mkdir -p /opt/cni/bin
220-
$ sudo tar -C /opt/cni/bin -xzf cni-plugins.tgz
221-
```
222-
Also, ensure your Linux operating system distribution has been configured to allow container traffic through the bridge network to be routed via iptables. These tunables can be set as follows:
221+
**Installing CNI plugins**
223222

223+
[//]: # (TODO: Replace this with containernetworking-plugins Ubuntu package)
224+
225+
```shell
226+
$ curl -L -o cni-plugins.tgz https://github.com/containernetworking/plugins/releases/download/v0.8.6/cni-plugins-linux-amd64-v0.8.6.tgz
227+
$ sudo mkdir -p /opt/cni/bin
228+
$ sudo tar -C /opt/cni/bin -xzf cni-plugins.tgz
224229
```
230+
231+
Also, ensure your Linux operating system distribution has been configured to allow container traffic through the bridge network to be routed via iptables.
232+
These tunables can be set as follows:
233+
234+
```shell
225235
$ echo 1 > /proc/sys/net/bridge/bridge-nf-call-arptables
226236
$ echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables
227237
$ echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
228238
```
229239
To preserve these settings on startup of a nomad client node, add a file including the following to `/etc/sysctl.d/` or remove the file your Linux distribution puts in that directory.
230240

231-
```
241+
```toml
232242
net.bridge.bridge-nf-call-arptables = 1
233243
net.bridge.bridge-nf-call-ip6tables = 1
234244
net.bridge.bridge-nf-call-iptables = 1
@@ -238,10 +248,10 @@ net.bridge.bridge-nf-call-iptables = 1
238248

239249
nomad supports both **static** and **dynamic** port mapping.
240250

241-
1. **Static ports**
251+
### Static ports
242252

243253
Static port mapping can be added in the `network` stanza.
244-
```
254+
```hcl
245255
network {
246256
mode = "bridge"
247257
port "lb" {
@@ -253,10 +263,10 @@ network {
253263
Here, `host` port `8889` is mapped to `container` port `8889`.<br/>
254264
**NOTE**: static ports are usually not recommended, except for `system` or specialized jobs like load balancers.
255265

256-
2. **Dynamic ports**
266+
### Dynamic ports
257267

258268
Dynamic port mapping is also enabled in the `network` stanza.
259-
```
269+
```hcl
260270
network {
261271
mode = "bridge"
262272
port "http" {
@@ -266,11 +276,13 @@ network {
266276
```
267277
Here, nomad will allocate a dynamic port on the `host` and that port will be mapped to `8080` in the container.
268278

269-
You can also read more about `network stanza` in the [`nomad official documentation`](https://www.nomadproject.io/docs/job-specification/network)
279+
You can also read more about `network stanza` in the [`nomad official documentation`](https://www.nomadproject.io/docs/job-specification/network).
270280

271281
## Service discovery
272282

273-
Nomad schedules workloads of various types across a cluster of generic hosts. Because of this, placement is not known in advance and you will need to use service discovery to connect tasks to other services deployed across your cluster. Nomad integrates with Consul to provide service discovery and monitoring.
283+
Nomad schedules workloads of various types across a cluster of generic hosts.
284+
Because of this, placement is not known in advance, and you will need to use service discovery to connect tasks to other services deployed across your cluster.
285+
Nomad integrates with Consul to provide service discovery and monitoring.
274286

275287
A [`service`](https://www.nomadproject.io/docs/job-specification/service) stanza can be added to your job spec, to enable service discovery.
276288

@@ -280,34 +292,36 @@ The service stanza instructs Nomad to register a service with Consul.
280292

281293
If you are running the tests locally, use the [`vagrant VM`](Vagrantfile) provided in the repository.
282294

283-
```
295+
```shell
284296
$ vagrant up
285297
$ vagrant ssh containerd-linux
286298
$ sudo make test
287299
```
288300
**NOTE**: These are destructive tests and can leave the system in a changed state.<br/>
289301
It is highly recommended to run these tests either as part of a CI/CD system e.g. circleci or on
290-
a immutable infrastructure e.g vagrant VMs.
302+
an immutable infrastructure e.g. vagrant VMs.
291303

292304
You can also run an individual test by specifying the test name. e.g.
293305

294-
```
306+
```shell
295307
$ cd tests
296308
$ sudo ./run_tests.sh 001-test-redis.sh
297309
```
298310

299311
## Cleanup
300-
```
301-
make clean
312+
313+
```shell
314+
$ make clean
302315
```
303316
This will delete your binary: `containerd-driver`
304317

305-
```
306-
vagrant destroy
318+
```shell
319+
$ vagrant destroy
307320
```
308321
This will destroy your vagrant VM.
309322

310323
## Currently supported environments
324+
311325
Ubuntu (>= 16.04)
312326

313327
## License

0 commit comments

Comments
 (0)