@@ -18,7 +18,7 @@ jailer --id <id> \
1818 --exec-file < exec_file> \
1919 --uid < uid> \
2020 --gid < gid> \
21- [--parent-cgroup < relative_path > ] \
21+ [--parent-cgroup < parent_cgroup > ] \
2222 [--cgroup-version < cgroup_version> ] \
2323 [--cgroup < cgroup> ] \
2424 [--chroot-base-dir < chroot_base> ] \
@@ -29,47 +29,50 @@ jailer --id <id> \
2929 [--...extra arguments for Firecracker]
3030```
3131
32- - ` id ` is the unique VM identification string, which may contain alphanumeric
33- characters and hyphens. The maximum ` id ` length is currently 64 characters.
34- - ` exec_file ` is the path to the Firecracker binary that will be exec-ed by the
35- jailer. The filename must include the string ` firecracker ` . This is enforced
36- because the interaction with the jailer is Firecracker specific.
37- - ` uid ` and ` gid ` are the uid and gid the jailer switches to as it execs the
38- target binary.
39- - ` parent-cgroup ` is used to allow the placement of microvm cgroups in custom
32+ - ` --id ` specifies the unique VM identification string, which may contain
33+ alphanumeric characters and hyphens. The maximum length is currently 64
34+ characters.
35+ - ` --exec-file ` specifies the path to the Firecracker binary that will be
36+ exec-ed by the jailer. The filename must include the string ` firecracker ` .
37+ This is enforced because the interaction with the jailer is Firecracker
38+ specific.
39+ - ` --uid ` and ` --gid ` specify the uid and gid the jailer switches to as it execs
40+ the target binary.
41+ - ` --parent-cgroup ` is used to allow the placement of microvm cgroups in custom
4042 nested hierarchies. By specifying this parameter, the jailer will create a new
41- cgroup named ` id ` for the microvm in the ` <cgroup_base>/<parent_cgroup> `
42- subfolder. ` cgroup_base ` is the cgroup controller root for ` cgroup v1 ` (e.g.
43+ cgroup named ` <id> ` for the microvm in the ` <cgroup_base>/<parent_cgroup> `
44+ subfolder. ` < cgroup_base> ` is the cgroup controller root for ` cgroup v1 ` (e.g.
4345 ` /sys/fs/cgroup/cpu ` ) or the unified controller hierarchy for ` cgroup v2 `
4446 (e.g. ` /sys/fs/cgroup/unified ` ). ` <parent_cgroup> ` is a relative path within
4547 that hierarchy. For example, if ` --parent-cgroup all_uvms/external_uvms ` is
4648 specified, the jailer will write all cgroup parameters specified through
4749 ` --cgroup ` in ` /sys/fs/cgroup/<controller_name>/all_uvms/external_uvms/<id> ` .
48- By default, the parent cgroup is ` exec-file ` . If there are no ` --cgroup `
50+ By default, the parent cgroup is the filename of ` <exec_file> ` , which will be
51+ henceforth referred to as ` <exec_file_name> ` . If there are no ` --cgroup `
4952 parameters specified and ` --group-version=2 ` was passed, then the jailer will
5053 move the process to the specified cgroup.
51- - ` cgroup-version ` is used to select which type of cgroup hierarchy to use for
54+ - ` -- cgroup-version` is used to select which type of cgroup hierarchy to use for
5255 the creation of cgroups. The default value is "1" which means that cgroups
53- specified with the ` cgroup ` argument will be created within a v1 hierarchy.
54- Supported options are "1" for cgroup-v1 and "2" for cgroup-v2.
55- - ` cgroup ` cgroups can be passed to the jailer to let it set the values when the
56- microVM process is spawned. The ` --cgroup ` argument must follow this format:
56+ specified with ` -- cgroup` will be created within a v1 hierarchy. Supported
57+ options are "1" for cgroup-v1 and "2" for cgroup-v2.
58+ - ` -- cgroup` can be passed to the jailer to let it set the values when the
59+ microVM process is spawned. The argument must follow this format:
5760 ` <cgroup_file>=<value> ` (e.g ` cpuset.cpus=0 ` ). This argument can be used
5861 multiple times to set multiple cgroups. This is useful to avoid providing
5962 privileged permissions to another process for setting the cgroups before or
6063 after the jailer is executed. The ` --cgroup ` flag can help as well to set
6164 Firecracker process cgroups before the VM starts running, with no need to
6265 create the entire cgroup hierarchy manually (which requires privileged
6366 permissions).
64- - ` chroot_base ` represents the base folder where chroot jails are built. The
65- default is ` /srv/jailer ` .
66- - ` netns ` represents the path to a network namespace handle. If present, the
67+ - ` --chroot-base-dir ` specifies the base folder where chroot jails are built.
68+ The default is ` /srv/jailer ` .
69+ - ` -- netns` specifies the path to a network namespace handle. If present, the
6770 jailer will use this to join the associated network namespace.
68- - For extra security and control over resource usage, ` resource-limit ` can be
69- used to set bounds to the process resources. The ` --resource-limit ` argument
70- must follow this format: ` <resource>=<value> ` (e.g ` no-file=1024 ` ) and can be
71- used multiple times to set multiple bounds. Current available resources that
72- can be limited using this argument are:
71+ - For extra security and control over resource usage, ` -- resource-limit` can be
72+ used to set bounds to the process resources. The argument must follow this
73+ format: ` <resource>=<value> ` (e.g ` no-file=1024 ` ) and can be used multiple
74+ times to set multiple bounds. Current available resources that can be limited
75+ using this argument are:
7376 - ` fsize ` : The maximum size in bytes for files created by the process.
7477 - ` no-file ` : Specifies a value one greater than the maximum file descriptor
7578 number that can be opened by this process.
@@ -80,13 +83,13 @@ Here is an example on how to set multiple resource limits using this argument:
8083--resource-limit fsize=250000000 --resource-limit no-file=1024
8184```
8285
83- - When present, the ` --daemonize ` flag causes the jailer to call ` setsid() ` and
84- redirect all three standard I/O file descriptors to ` /dev/null ` .
85- - When present, the ` --new-pid-ns ` flag causes the jailer to spawn the provided
86- binary into a new PID namespace. It makes use of the libc ` clone() ` function
87- with the ` CLONE_NEWPID ` flag. As a result, the jailer and the process running
88- the exec file have different PIDs. The PID of the child process is stored in
89- the jail root directory inside ` <exec_file_name>.pid ` .
86+ - When present, ` --daemonize ` causes the jailer to call ` setsid() ` and redirect
87+ all three standard I/O file descriptors to ` /dev/null ` .
88+ - When present, ` --new-pid-ns ` causes the jailer to spawn the provided binary
89+ into a new PID namespace. It makes use of the libc ` clone() ` function with the
90+ ` CLONE_NEWPID ` flag. As a result, the jailer and the process running the exec
91+ file have different PIDs. The PID of the child process is stored in the jail
92+ root directory inside ` <exec_file_name>.pid ` .
9093- The jailer adheres to the "end of command options" convention, meaning all
9194 parameters specified after ` -- ` are forwarded to Firecracker. For example,
9295 this can be paired with the ` --config-file ` Firecracker argument to specify a
@@ -99,23 +102,21 @@ Here is an example on how to set multiple resource limits using this argument:
99102
100103After starting, the Jailer goes through the following operations:
101104
102- - Validate ** all provided paths** and the VM ` id ` .
105+ - Validate ** all provided paths** and the VM ID .
103106- Close all open file descriptors based on ` /proc/<jailer-pid>/fd ` except input,
104107 output and error.
105108- Cleanup all environment variables received from the parent process.
106109- Create the ` <chroot_base>/<exec_file_name>/<id>/root ` folder, which will be
107- henceforth referred to as ` chroot_dir ` . ` exec_file_name ` is the last path
108- component of ` exec_file ` (for example, that would be ` firecracker ` for
109- ` /usr/bin/firecracker ` ). Nothing is done if the path already exists (it should
110- not, since ` id ` is supposed to be unique).
111- - Copy ` exec_file ` to
112- ` <chroot_base>/<exec_file_name>/<id>/root/<exec_file_name> ` . This ensures the
113- new process will not share memory with any other Firecracker process.
110+ henceforth referred to as ` <chroot_dir> ` . Nothing is done if the path already
111+ exists (it should not, since ` <id> ` is supposed to be unique).
112+ - Copy the file specified with ` --exec-file ` to ` <chroot_dir>/<exec_file_name> ` .
113+ This ensures the new process will not share memory with any other Firecracker
114+ process.
114115- Set resource bounds for current process and its children through
115116 ` --resource-limit ` argument, by calling ` setrlimit() ` system call with the
116117 specific resource argument. If no limits are provided, the jailer bounds
117118 ` no-file ` to a maximum default value of 2048.
118- - Create the ` cgroup ` sub-folders. The jailer can use either ` cgroup v1 ` or
119+ - Create the cgroup sub-folders. The jailer can use either ` cgroup v1 ` or
119120 ` cgroup v2 ` . On most systems, this is mounted by default in ` /sys/fs/cgroup `
120121 (should be mounted by the user otherwise). The jailer will parse
121122 ` /proc/mounts ` to detect where each of the controllers required in ` --cgroup `
@@ -125,14 +126,14 @@ After starting, the Jailer goes through the following operations:
125126 ` <cgroup_base>/<parent_cgroup>/<id>/tasks ` . Also, the value passed for each
126127 ` <cgroup_file> ` is written to the file.
127128- Call ` unshare() ` into a new mount namespace, use ` pivot_root() ` to switch the
128- old system root mount point with a new one base in ` chroot_dir ` , switch the
129+ old system root mount point with a new one base in ` < chroot_dir> ` , switch the
129130 current working directory to the new root, unmount the old root mount point,
130131 and call ` chroot ` into the current directory.
131132- Use ` mknod ` to create a ` /dev/net/tun ` equivalent inside the jail.
132133- Use ` mknod ` to create a ` /dev/kvm ` equivalent inside the jail.
133- - Use ` chown ` to change ownership of the ` chroot_dir ` (root path ` / ` as seen by
134- the jailed firecracker), ` /dev/net/tun ` , ` /dev/kvm ` . The ownership is changed
135- to the provided ` uid: gid ` .
134+ - Use ` chown ` to change ownership of the ` < chroot_dir> ` (root path ` / ` as seen
135+ by the jailed firecracker), ` /dev/net/tun ` , ` /dev/kvm ` . The ownership is
136+ changed to the provided ` < uid>:< gid> ` .
136137- If ` --netns <netns> ` is present, attempt to join the specified network
137138 namespace.
138139- If ` --daemonize ` is specified, call ` setsid() ` and redirect ` STDIN ` , ` STDOUT ` ,
@@ -147,8 +148,8 @@ After starting, the Jailer goes through the following operations:
147148 ` <exec_file_name> --id=<id> --start-time-us=<opaque> --start-time-cpu-us=<opaque> `
148149 (and also forward any extra arguments provided to the jailer after ` -- ` , as
149150 mentioned in the ** Jailer Usage** section), where:
150- - ` id ` : (` string ` ) - The ` id ` argument provided to jailer.
151- - ` opaque ` : (` number ` ) time calculated by the jailer that it spent doing its
151+ - ` <id> ` : (` string ` ) - The ` <id> ` argument provided to jailer.
152+ - ` < opaque> ` : (` number ` ) time calculated by the jailer that it spent doing its
152153 work.
153154
154155## Example Run and Notes
@@ -191,7 +192,7 @@ It’s worth noting that, whenever a folder already exists, nothing will be done
191192and we move on to the next directory that needs to be created. This should only
192193happen for the common ` firecracker ` subfolder (but, as for creating the chroot
193194path before, we do not issue an error if folders directly associated with the
194- supposedly unique ` id ` already exist).
195+ supposedly unique ` <id> ` already exist).
195196
196197The jailer then writes the current pid to
197198` /sys/fs/cgroup/cpuset/firecracker/551e7604-e35c-42b3-b825-416853441234/tasks ` ,
@@ -228,16 +229,16 @@ call `chown(“/dev/net/tun”, 123, 100)`, so Firecracker can use it after drop
228229privileges. This is required to use multiple TAP interfaces when running jailed.
229230Do the same for ` /dev/kvm ` .
230231
231- Change ownership of ` <chroot_dir> ` to ` uid: gid ` so that Firecracker can create
232- its API socket there.
232+ Change ownership of ` <chroot_dir> ` to ` < uid>:< gid> ` so that Firecracker can
233+ create its API socket there.
233234
234235Since the ` --daemonize ` flag is present, call ` setsid() ` to join a new session,
235236a new process group, and to detach from the controlling terminal. Then, redirect
236237standard file descriptors to ` /dev/null ` by calling ` dup2(dev_null_fd, STDIN) ` ,
237238` dup2(dev_null_fd, STDOUT) ` , and ` dup2(dev_null_fd, STDERR) ` . Close
238239` dev_null_fd ` , because it is no longer necessary.
239240
240- Finally, the jailer switches the ` uid ` to ` 123 ` , and ` gid ` to ` 100 ` , and execs
241+ Finally, the jailer switches the uid to ` 123 ` , and gid to ` 100 ` , and execs
241242
242243``` console
243244./firecracker \
0 commit comments