Skip to content

Commit cc98447

Browse files
cgwalterscgwalters
authored
Merge pull request #422 from cgwalters/build-guidance
docs: Add a new "build guidance" section
2 parents 60552ee + 4c4f471 commit cc98447

File tree

3 files changed

+289
-0
lines changed

3 files changed

+289
-0
lines changed

‎docs/src/SUMMARY.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,11 @@
66

77
- [Installation](installation.md)
88

9+
# Building images
10+
11+
- [Building images](building/guidance.md)
12+
- [Users, groups, SSH keys](building/users-and-groups.md)
13+
914
# Using bootc
1015

1116
- [Upgrade and rollback](upgrades.md)

‎docs/src/building/guidance.md

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Generic guidance for building images
2+
3+
The bootc project intends to be operating system and distribution independent as possible,
4+
similar to its related projects [podman](http://podman.io/) and [systemd](https://systemd.io/),
5+
etc.
6+
7+
The recommendations for creating bootc-compatible images will in general need to
8+
be owned by the OS/distribution - in particular the ones who create the default
9+
bootc base image(s). However, some guidance is very generic to most Linux
10+
systems (and bootc only supports Linux).
11+
12+
Let's however restate a base goal of this project:
13+
14+
> The original Docker container model of using "layers" to model
15+
> applications has been extremely successful. This project
16+
> aims to apply the same technique for bootable host systems - using
17+
> standard OCI/Docker containers as a transport and delivery format
18+
> for base operating system updates.
19+
20+
Every tool and technique for creating application base images
21+
should apply to the host Linux OS as much as possible.
22+
23+
## Installing software
24+
25+
For package management tools like `apt`, `dnf`, `zypper` etc.
26+
(generically, `$pkgsystem`) it is very much expected that
27+
the pattern of
28+
29+
`RUN $pkgsystem install somepackage && $pkgsystem clean all`
30+
31+
type flow Just Works here - the same way as it does
32+
"application" container images. This pattern is really how
33+
Docker got started.
34+
35+
There's not much special to this that doesn't also apply
36+
to application containers; but see below.
37+
38+
## systemd units
39+
40+
The model that is most popular with the Docker/OCI world
41+
is "microservice" style containers with the application as
42+
pid 1, isolating the applications from each other and
43+
from the host system - as opposed to "system containers"
44+
which run an init system like systemd, typically also
45+
SSH and often multiple logical "application" components
46+
as part of the same container.
47+
48+
The bootc project generally expects systemd as pid 1,
49+
and if you embed software in your derived image, the
50+
default would then be that that software is initially
51+
launched via a systemd unit.
52+
53+
```
54+
RUN dnf -y install postgresql
55+
```
56+
57+
Would typically also carry a systemd unit, and that
58+
service will be launched the same way as it would
59+
on a package-based system.
60+
61+
## Users and groups
62+
63+
Note that the above `postgresql` today will allocate a user;
64+
this leads to the topic of [users, groups and SSH keys](users-and-groups.md).
65+
66+
## Configuration
67+
68+
A key aspect of choosing a bootc-based operating system model
69+
is that *code* and *configuration* can be strictly "lifecycle bound"
70+
together in exactly the same way.
71+
72+
(Today, that's by including the configuration into the base
73+
container image; however a future enhancement for bootc
74+
will also support dynamically-injected ConfigMaps, similar
75+
to kubelet)
76+
77+
You can add configuration files to the same places they're
78+
expected by typical package systems on Debian/Fedora/Arch
79+
etc. and others - in `/usr` (preferred where possible)
80+
or `/etc`. systemd has long advocated and supported
81+
a model where `/usr` (e.g. `/usr/lib/systemd/system`)
82+
contains content owned by the operating system image.
83+
84+
`/etc` is machine-local state. However, per [filesystem.md](../filesystem.md)
85+
it's important to note that the underlying OSTree
86+
system performs a 3-way merge of `/etc`, so changes you
87+
make in the container image to e.g. `/etc/postgresql.conf`
88+
will be applied on update, assuming it is not modified
89+
locally.
90+

‎docs/src/building/users-and-groups.md

Lines changed: 194 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,194 @@
1+
2+
# Users and groups
3+
4+
This is one of the more complex topics. Generally speaking, bootc has nothing to
5+
do directly with configuring users or groups; it is a generic OS
6+
update/configuration mechanism. (There is currently just one small exception in
7+
that `bootc install` has a special case `--root-ssh-authorized-keys` argument,
8+
but it's very much optional).
9+
10+
## Generic base images
11+
12+
Commonly OS/distribution base images will be generic, i.e.
13+
without any configuration. It is *very strongly recommended*
14+
to avoid hardcoded passwords and ssh keys with publicly-available
15+
private keys (as Vagrant does) in generic images.
16+
17+
### Injecting SSH keys via systemd credentials
18+
19+
The systemd project has documentation for [credentials](https://systemd.io/CREDENTIALS/)
20+
which can be used in some environments to inject a root
21+
password or SSH authorized_keys. For many cases, this
22+
is a best practice.
23+
24+
At the time of this writing this relies on SMBIOS which
25+
is mainly configurable in local virtualization environments.
26+
(qemu).
27+
28+
### Injecting users and SSH keys via cloud-init, etc.
29+
30+
Many IaaS and virtualization systems are oriented towards a "metadata server"
31+
(see e.g. [AWS instance metadata](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html))
32+
that are commonly processed by software such as [cloud-init](https://cloud-init.io/)
33+
or [Ignition](https://github.com/coreos/ignition) or equivalent.
34+
35+
The base image you're using may include such software, or you
36+
can install it in your own derived images.
37+
38+
In this model, SSH configuration is managed outside of the bootable
39+
image. See e.g. [GCP oslogin](https://cloud.google.com/compute/docs/oslogin/)
40+
for an example of this where operating system identities are linked
41+
to the underlying Google accounts.
42+
43+
### Adding users and credentials via custom logic (container or unit)
44+
45+
Of course, systems like `cloud-init` are not privileged; you
46+
can inject any logic you want to manage credentials via
47+
e.g. a systemd unit (which may launch a container image)
48+
that manages things however you prefer. Commonly,
49+
this would be a custom network-hosted source. For example,
50+
[FreeIPA](https://www.freeipa.org/page/Main_Page).
51+
52+
Another example in a Kubernetes-oriented infrastructure would
53+
be a container image that fetches desired authentication
54+
credentials from a [CRD](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/)
55+
hosted in the API server. (To do things like this
56+
it's suggested to reuse the kubelet credentials)
57+
58+
### Adding users and credentials statically in the container build
59+
60+
Relative to package-oriented systems, a new ability is to inject
61+
users and credentials as part of a derived build:
62+
63+
```dockerfile
64+
RUN useradd someuser
65+
```
66+
67+
However, it is important to understand some issues with the default
68+
`shadow-utils` implementation of `useradd`:
69+
70+
First, typically user/group IDs are allocated dynamically, and this can result in "drift" (see below).
71+
72+
#### User and group home directories and `/var`
73+
74+
For systems configured with persistent `/home` → `/var/home`, any changes to `/var` made
75+
in the container image after initial installation *will not be applied on subsequent updates*. If for example you inject `/var/home/someuser/.ssh/authorized_keys`
76+
into a container build, existing systems will *not* get the updated authorized keys file.
77+
78+
#### Using DynamicUser=yes for systemd units
79+
80+
For "system" users it's strongly recommended to use systemd [DynamicUser=yes](https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html#DynamicUser=) where
81+
possible.
82+
83+
This is significantly better than the pattern of allocating users/groups
84+
at "package install time" (e.g. [Fedora package user/group guidelines](https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/)) because
85+
it avoids potential UID/GID drift (see below).
86+
87+
#### Using systemd-sysusers
88+
89+
See [systemd-sysusers](https://www.freedesktop.org/software/systemd/man/latest/systemd-sysusers.html). For example in your derived build:
90+
91+
```
92+
COPY mycustom-user.conf /usr/lib/sysusers.d
93+
```
94+
95+
A key aspect of how this works is that `sysusers` will make changes
96+
to the traditional `/etc/passwd` file as necessary on boot. If
97+
`/etc` is persistent, this can avoid uid/gid drift (but
98+
in the general case it does mean that uid/gid allocation can
99+
depend on how a specific machine was upgraded over time).
100+
101+
#### Using systemd JSON user records
102+
103+
See [JSON user records](https://systemd.io/USER_RECORD/). Unlike `sysusers`,
104+
the canonical state for these live in `/usr` - if a subsequent
105+
image drops a user record, then it will also vanish
106+
from the system - unlike `sysusers.d`.
107+
108+
#### nss-altfiles
109+
110+
The [nss-altfiles](https://github.com/aperezdc/nss-altfiles) project
111+
(long) predates systemd JSON user records. It aims to help split
112+
"system" users into `/usr/lib/passwd` and `/usr/lib/group`. It's
113+
very important to understand that this aligns with the way
114+
the OSTree project handles the "3 way merge" for `/etc` as it
115+
relates to `/etc/passwd`. Currently, if the `/etc/passwd` file is
116+
modified in any way on the local system, then subsequent changes
117+
to `/etc/passwd` in the container image *will not be applied*.
118+
119+
Some base images may have `nss-altfiles` enabled by default;
120+
this is currently the case for base images built by
121+
[rpm-ostree](https://github.com/coreos/rpm-ostree).
122+
123+
Commonly, base images will have some "system" users pre-allocated
124+
and managed via this file again to avoid uid/gid drift.
125+
126+
In a derived container build, you can also append users
127+
to `/usr/lib/passwd` for example. (At the time of this
128+
writing there is no command line to do so though).
129+
130+
Typically it is more preferable to use `sysusers.d`
131+
or `DynamicUser=yes`.
132+
133+
### Machine-local state for users
134+
135+
At this point, it is important to understand the [filesystem](filesystem.md)
136+
layout - the default is up to the base image.
137+
138+
The default Linux concept of a user has data stored in both `/etc` (`/etc/passwd`, `/etc/shadow` and groups)
139+
and `/home`. The choice for how these work is up to the base image, but
140+
a common default for generic base images is to have both be machine-local persistent state.
141+
In this model `/home` would be a symlink to `/var/home/someuser`.
142+
143+
But it is also valid to default to having e.g. `/home` be a `tmpfs`
144+
to ensure user data is cleaned up across reboots (and this pairs particularly
145+
well with a transient `/etc` as well).
146+
147+
#### Injecting users and SSH keys via at system provisioning time
148+
149+
For base images where `/etc` and `/var` are configured to persist by default, it
150+
will then be generally supported to inject users via "installers" such
151+
as [Anaconda](https://github.com/rhinstaller/anaconda/) (interactively or
152+
via kickstart) or any others.
153+
154+
Typically generic installers such as this are designed for "one time bootstrap"
155+
and again then the configuration becomes mutable machine-local state
156+
that can be changed "day 2" via some other mechanism.
157+
158+
The simple case is a user with a password - typically the installer helps
159+
set the initial password, but to change it there is a different in-system
160+
tool (such as `passwd` or a GUI as part of [Cockpit](https://cockpit-project.org/), GNOME/KDE/etc).
161+
162+
It is intended that these flows work equivalently in a bootc-compatible
163+
system, to support users directly installing "generic" base images, without
164+
requiring changes to the tools above.
165+
166+
### UID/GID drift
167+
168+
Ultimately the `/etc/passwd` and similar files are a mapping
169+
between names and numeric identifiers. A problem then becomes
170+
when this mapping is dynamic and mixed with "stateless"
171+
container image builds.
172+
173+
For example today the CentOS Stream 9 `postgresql` package
174+
allocates a [static uid of `26`](https://gitlab.com/redhat/centos-stream/rpms/postgresql/-/blob/a03cf81d4b9a77d9150a78949269ae52a0027b54/postgresql.spec#L847).
175+
176+
This means that
177+
```
178+
RUN dnf -y install postgresql
179+
```
180+
181+
will always result in a change to `/etc/passwd` that allocates uid 26
182+
and data in `/var/lib/postgres` will always be owned by that UID.
183+
184+
However in contrast, the cockpit project allocates
185+
[a floating cockpit-ws user](https://gitlab.com/redhat/centos-stream/rpms/cockpit/-/blob/1909236ad28c7d93238b8b3b806ecf9c4feb7e46/cockpit.spec#L506).
186+
187+
This means that each container image build (without additional work)
188+
may (due to RPM installation ordering or other reasons) result
189+
in the uid changing.
190+
191+
This can be a problem if that user maintains persistent state.
192+
Such cases are best handled by being converted to use `sysusers.d`
193+
(see [Fedora change](https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format)) - or again even better, using `DynamicUser=yes` (see above).
194+

0 commit comments

Comments
 (0)