-
Notifications
You must be signed in to change notification settings - Fork 8.1k
engine/security/rootless: split to multiple pages ; remove outdated/duplicated info #23302
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
bc4ed83
engine/security/rootless: split to multiple pages
AkihiroSuda 8e4e838
engine/security/rootless: remove outdated/duplicated info
AkihiroSuda 061ed75
engine/security/rootless: fix reviewdog errors
AkihiroSuda 9a04913
engine/security/rootless: fix typo
AkihiroSuda bb8ab92
Apply suggestions from code review
aevesdocker File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,147 @@ | ||
| --- | ||
| description: Run the Docker daemon as a non-root user (Rootless mode) | ||
| keywords: security, namespaces, rootless | ||
| title: Rootless mode | ||
| weight: 10 | ||
| --- | ||
|
|
||
| Rootless mode allows running the Docker daemon and containers as a non-root | ||
| user to mitigate potential vulnerabilities in the daemon and | ||
| the container runtime. | ||
|
|
||
| Rootless mode does not require root privileges even during the installation of | ||
| the Docker daemon, as long as the [prerequisites](#prerequisites) are met. | ||
|
|
||
| ## How it works | ||
|
|
||
| Rootless mode executes the Docker daemon and containers inside a user namespace. | ||
| This is similar to [`userns-remap` mode](../userns-remap.md), except that | ||
| with `userns-remap` mode, the daemon itself is running with root privileges, | ||
| whereas in rootless mode, both the daemon and the container are running without | ||
| root privileges. | ||
|
|
||
| Rootless mode does not use binaries with `SETUID` bits or file capabilities, | ||
| except `newuidmap` and `newgidmap`, which are needed to allow multiple | ||
| UIDs/GIDs to be used in the user namespace. | ||
|
|
||
|
|
||
| ## Prerequisites | ||
|
|
||
| - You must install `newuidmap` and `newgidmap` on the host. These commands | ||
| are provided by the `uidmap` package on most distributions. | ||
|
|
||
| - `/etc/subuid` and `/etc/subgid` should contain at least 65,536 subordinate | ||
| UIDs/GIDs for the user. In the following example, the user `testuser` has | ||
| 65,536 subordinate UIDs/GIDs (231072-296607). | ||
|
|
||
| ```console | ||
| $ id -u | ||
| 1001 | ||
| $ whoami | ||
| testuser | ||
| $ grep ^$(whoami): /etc/subuid | ||
| testuser:231072:65536 | ||
| $ grep ^$(whoami): /etc/subgid | ||
| testuser:231072:65536 | ||
| ``` | ||
|
|
||
| The `dockerd-rootless-setuptool.sh install` script (see following) automatically shows help | ||
| when the prerequiresites are not satisfied. | ||
AkihiroSuda marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ## Install | ||
|
|
||
| > [!NOTE] | ||
| > | ||
| > If the system-wide Docker daemon is already running, consider disabling it: | ||
| >```console | ||
| >$ sudo systemctl disable --now docker.service docker.socket | ||
| >$ sudo rm /var/run/docker.sock | ||
| >``` | ||
| > Should you choose not to shut down the `docker` service and socket, you will need to use the `--force` | ||
| > parameter in the next section. There are no known issues, but until you shutdown and disable you're | ||
| > still running rootful Docker. | ||
|
|
||
| {{< tabs >}} | ||
| {{< tab name="With packages (RPM/DEB)" >}} | ||
|
|
||
| If you installed Docker 20.10 or later with [RPM/DEB packages](/engine/install), you should have `dockerd-rootless-setuptool.sh` in `/usr/bin`. | ||
|
|
||
| Run `dockerd-rootless-setuptool.sh install` as a non-root user to set up the daemon: | ||
|
|
||
| ```console | ||
| $ dockerd-rootless-setuptool.sh install | ||
| [INFO] Creating /home/testuser/.config/systemd/user/docker.service | ||
| ... | ||
| [INFO] Installed docker.service successfully. | ||
| [INFO] To control docker.service, run: `systemctl --user (start|stop|restart) docker.service` | ||
| [INFO] To run docker.service on system startup, run: `sudo loginctl enable-linger testuser` | ||
|
|
||
| [INFO] Creating CLI context "rootless" | ||
| Successfully created context "rootless" | ||
| [INFO] Using CLI context "rootless" | ||
| Current context is now "rootless" | ||
|
|
||
| [INFO] Make sure the following environment variable(s) are set (or add them to ~/.bashrc): | ||
| export PATH=/usr/bin:$PATH | ||
|
|
||
| [INFO] Some applications may require the following environment variable too: | ||
| export DOCKER_HOST=unix:///run/user/1000/docker.sock | ||
| ``` | ||
|
|
||
| If `dockerd-rootless-setuptool.sh` is not present, you may need to install the `docker-ce-rootless-extras` package manually, e.g., | ||
|
|
||
| ```console | ||
| $ sudo apt-get install -y docker-ce-rootless-extras | ||
| ``` | ||
|
|
||
| {{< /tab >}} | ||
| {{< tab name="Without packages" >}} | ||
|
|
||
| If you do not have permission to run package managers like `apt-get` and `dnf`, | ||
| consider using the installation script available at [https://get.docker.com/rootless](https://get.docker.com/rootless). | ||
| Since static packages are not available for `s390x`, hence it is not supported for `s390x`. | ||
|
|
||
| ```console | ||
| $ curl -fsSL https://get.docker.com/rootless | sh | ||
| ... | ||
| [INFO] Creating /home/testuser/.config/systemd/user/docker.service | ||
| ... | ||
| [INFO] Installed docker.service successfully. | ||
| [INFO] To control docker.service, run: `systemctl --user (start|stop|restart) docker.service` | ||
| [INFO] To run docker.service on system startup, run: `sudo loginctl enable-linger testuser` | ||
|
|
||
| [INFO] Creating CLI context "rootless" | ||
| Successfully created context "rootless" | ||
| [INFO] Using CLI context "rootless" | ||
| Current context is now "rootless" | ||
|
|
||
| [INFO] Make sure the following environment variable(s) are set (or add them to ~/.bashrc): | ||
| export PATH=/home/testuser/bin:$PATH | ||
|
|
||
| [INFO] Some applications may require the following environment variable too: | ||
| export DOCKER_HOST=unix:///run/user/1000/docker.sock | ||
| ``` | ||
|
|
||
| The binaries will be installed at `~/bin`. | ||
|
|
||
| {{< /tab >}} | ||
| {{< /tabs >}} | ||
|
|
||
| Run `docker info` to confirm that the `docker` client is connecting to the Rootless daemon: | ||
| ```console | ||
| $ docker info | ||
| Client: Docker Engine - Community | ||
| Version: 28.3.3 | ||
| Context: rootless | ||
| ... | ||
| Server: | ||
| ... | ||
| Security Options: | ||
| seccomp | ||
| Profile: builtin | ||
| rootless | ||
| cgroupns | ||
| ... | ||
| ``` | ||
|
|
||
| See [Troubleshooting](./troubleshoot.md) if you faced an error. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,185 @@ | ||
| --- | ||
| description: Tips for the Rootless mode | ||
| keywords: security, namespaces, rootless | ||
| title: Tips | ||
| weight: 20 | ||
| --- | ||
|
|
||
| ## Advanced Usage | ||
aevesdocker marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ### Daemon | ||
|
|
||
| {{< tabs >}} | ||
| {{< tab name="With systemd (Highly recommended)" >}} | ||
|
|
||
| The systemd unit file is installed as `~/.config/systemd/user/docker.service`. | ||
|
|
||
| Use `systemctl --user` to manage the lifecycle of the daemon: | ||
|
|
||
| ```console | ||
| $ systemctl --user start docker | ||
| ``` | ||
|
|
||
| To launch the daemon on system startup, enable the systemd service and lingering: | ||
|
|
||
| ```console | ||
| $ systemctl --user enable docker | ||
| $ sudo loginctl enable-linger $(whoami) | ||
| ``` | ||
|
|
||
| Starting Rootless Docker as a systemd-wide service (`/etc/systemd/system/docker.service`) | ||
| is not supported, even with the `User=` directive. | ||
|
|
||
| {{< /tab >}} | ||
| {{< tab name="Without systemd" >}} | ||
|
|
||
| To run the daemon directly without systemd, you need to run `dockerd-rootless.sh` instead of `dockerd`. | ||
|
|
||
| The following environment variables must be set: | ||
| - `$HOME`: the home directory | ||
| - `$XDG_RUNTIME_DIR`: an ephemeral directory that is only accessible by the expected user, e,g, `~/.docker/run`. | ||
| The directory should be removed on every host shutdown. | ||
| The directory can be on tmpfs, however, should not be under `/tmp`. | ||
| Locating this directory under `/tmp` might be vulnerable to TOCTOU attack. | ||
|
|
||
| {{< /tab >}} | ||
| {{< /tabs >}} | ||
|
|
||
| Remarks about directory paths: | ||
aevesdocker marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| - The socket path is set to `$XDG_RUNTIME_DIR/docker.sock` by default. | ||
| `$XDG_RUNTIME_DIR` is typically set to `/run/user/$UID`. | ||
| - The data dir is set to `~/.local/share/docker` by default. | ||
| The data dir should not be on NFS. | ||
| - The daemon config dir is set to `~/.config/docker` by default. | ||
| This directory is different from `~/.docker` that is used by the client. | ||
|
|
||
| ### Client | ||
|
|
||
| Since Docker Engine v23.0, `dockerd-rootless-setuptool.sh install` automatically configures | ||
| the `docker` CLI to use the `rootless` context. | ||
|
|
||
| Prior to Docker Engine v23.0, a user had to specify either the socket path or the CLI context explicitly. | ||
|
|
||
| To specify the socket path using `$DOCKER_HOST`: | ||
|
|
||
| ```console | ||
| $ export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/docker.sock | ||
| $ docker run -d -p 8080:80 nginx | ||
| ``` | ||
|
|
||
| To specify the CLI context using `docker context`: | ||
|
|
||
| ```console | ||
| $ docker context use rootless | ||
| rootless | ||
| Current context is now "rootless" | ||
| $ docker run -d -p 8080:80 nginx | ||
| ``` | ||
|
|
||
| ## Best practices | ||
|
|
||
| ### Rootless Docker in Docker | ||
|
|
||
| To run Rootless Docker inside "rootful" Docker, use the `docker:<version>-dind-rootless` | ||
| image instead of `docker:<version>-dind`. | ||
|
|
||
| ```console | ||
| $ docker run -d --name dind-rootless --privileged docker:25.0-dind-rootless | ||
| ``` | ||
|
|
||
| The `docker:<version>-dind-rootless` image runs as a non-root user (UID 1000). | ||
| However, `--privileged` is required for disabling seccomp, AppArmor, and mount | ||
| masks. | ||
|
|
||
| ### Expose Docker API socket through TCP | ||
|
|
||
| To expose the Docker API socket through TCP, you need to launch `dockerd-rootless.sh` | ||
| with `DOCKERD_ROOTLESS_ROOTLESSKIT_FLAGS="-p 0.0.0.0:2376:2376/tcp"`. | ||
|
|
||
| ```console | ||
| $ DOCKERD_ROOTLESS_ROOTLESSKIT_FLAGS="-p 0.0.0.0:2376:2376/tcp" \ | ||
| dockerd-rootless.sh \ | ||
| -H tcp://0.0.0.0:2376 \ | ||
| --tlsverify --tlscacert=ca.pem --tlscert=cert.pem --tlskey=key.pem | ||
| ``` | ||
|
|
||
| ### Expose Docker API socket through SSH | ||
|
|
||
| To expose the Docker API socket through SSH, you need to make sure `$DOCKER_HOST` | ||
| is set on the remote host. | ||
|
|
||
| ```console | ||
| $ ssh -l <REMOTEUSER> <REMOTEHOST> 'echo $DOCKER_HOST' | ||
| unix:///run/user/1001/docker.sock | ||
| $ docker -H ssh://<REMOTEUSER>@<REMOTEHOST> run ... | ||
| ``` | ||
|
|
||
| ### Routing ping packets | ||
|
|
||
| On some distributions, `ping` does not work by default. | ||
|
|
||
| Add `net.ipv4.ping_group_range = 0 2147483647` to `/etc/sysctl.conf` (or | ||
| `/etc/sysctl.d`) and run `sudo sysctl --system` to allow using `ping`. | ||
|
|
||
| ### Exposing privileged ports | ||
|
|
||
| To expose privileged ports (< 1024), set `CAP_NET_BIND_SERVICE` on `rootlesskit` binary and restart the daemon. | ||
|
|
||
| ```console | ||
| $ sudo setcap cap_net_bind_service=ep $(which rootlesskit) | ||
| $ systemctl --user restart docker | ||
| ``` | ||
|
|
||
| Or add `net.ipv4.ip_unprivileged_port_start=0` to `/etc/sysctl.conf` (or | ||
| `/etc/sysctl.d`) and run `sudo sysctl --system`. | ||
|
|
||
| ### Limiting resources | ||
|
|
||
| Limiting resources with cgroup-related `docker run` flags such as `--cpus`, `--memory`, `--pids-limit` | ||
| is supported only when running with cgroup v2 and systemd. | ||
| See [Changing cgroup version](/manuals/engine/containers/runmetrics.md) to enable cgroup v2. | ||
|
|
||
| If `docker info` shows `none` as `Cgroup Driver`, the conditions are not satisfied. | ||
| When these conditions are not satisfied, rootless mode ignores the cgroup-related `docker run` flags. | ||
| See [Limiting resources without cgroup](#limiting-resources-without-cgroup) for workarounds. | ||
|
|
||
| If `docker info` shows `systemd` as `Cgroup Driver`, the conditions are satisfied. | ||
| However, typically, only `memory` and `pids` controllers are delegated to non-root users by default. | ||
|
|
||
| ```console | ||
| $ cat /sys/fs/cgroup/user.slice/user-$(id -u).slice/user@$(id -u).service/cgroup.controllers | ||
| memory pids | ||
| ``` | ||
|
|
||
| To allow delegation of all controllers, you need to change the systemd configuration as follows: | ||
|
|
||
| ```console | ||
| # mkdir -p /etc/systemd/system/[email protected] | ||
| # cat > /etc/systemd/system/[email protected]/delegate.conf << EOF | ||
| [Service] | ||
| Delegate=cpu cpuset io memory pids | ||
| EOF | ||
| # systemctl daemon-reload | ||
| ``` | ||
|
|
||
| > [!NOTE] | ||
| > | ||
| > Delegating `cpuset` requires systemd 244 or later. | ||
|
|
||
| #### Limiting resources without cgroup | ||
|
|
||
| Even when cgroup is not available, you can still use the traditional `ulimit` and [`cpulimit`](https://github.com/opsengine/cpulimit), | ||
| though they work in process-granularity rather than in container-granularity, | ||
| and can be arbitrarily disabled by the container process. | ||
|
|
||
| For example: | ||
|
|
||
| - To limit CPU usage to 0.5 cores (similar to `docker run --cpus 0.5`): | ||
| `docker run <IMAGE> cpulimit --limit=50 --include-children <COMMAND>` | ||
| - To limit max VSZ to 64MiB (similar to `docker run --memory 64m`): | ||
| `docker run <IMAGE> sh -c "ulimit -v 65536; <COMMAND>"` | ||
|
|
||
| - To limit max number of processes to 100 per namespaced UID 2000 | ||
| (similar to `docker run --pids-limit=100`): | ||
| `docker run --user 2000 --ulimit nproc=100 <IMAGE> <COMMAND>` | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.