Reduced warning on Sysbox being in alpha, per customer feedback.

ctalledo · nestybox · commit e812ca3e955f · 2019-11-20T22:44:35.000-08:00
Also, changed the trouble-shooting guide to place the shiftfs error
higher up as customers may likely stumble on it.
diff --git a/README.md b/README.md
@@ -53,9 +53,6 @@ create system containers with Docker as described below.
 
 ## Features
 
-**NOTE**: It's early days for Nestybox, so our system containers
-support a reduced set of features and use-cases at this time.
-
 Below is a list of features currently supported by Sysbox.
 
 ### System Container Deployment
@@ -141,9 +138,9 @@ stated above, see [here](docs/troubleshoot.md#upgrading-the-ubuntu-kernel)
 for suggestions on how to do this.
 
 Alternatively it's possible to use Sysbox with slightly older Ubuntu
-kernels, but doing so requires configuring Sysbox in "Docker
-userns-remap isolation mode". In this case you can run Sysbox on the
-following distros (without needing to upgrade the kernel):
+kernels, but doing so requires using system containers in [Docker userns-remap isolation mode](docs/usage.md#system-container-isolation-modes).
+In this case you can run Sysbox on the following distros (without
+needing to upgrade the kernel):
 
 -   Ubuntu 19.04 "Disco"
 -   Ubuntu 18.10 "Cosmic"
@@ -153,13 +150,13 @@ We plan to add support for more distros in the future.
 
 Here is a summary of the distro requirements:
 
-| System Container Isolation Mode | Required Distro & Kernel            |
-| ------------------------------- | ----------------------------------- |
-| Exclusive userns-remap          | Ubuntu 19.04 Disco (>= 5.0.0-21.22) |
-|                                 | Ubuntu 18.04 Bionic (5.0+ kernel)   |
-| Docker userns-remap             | Ubuntu 19.04 Disco                  |
-|                                 | Ubuntu 18.10 Cosmic                 |
-|                                 | Ubuntu 18.04 Bionic                 |
+| System Container Isolation Mode  | Required Distro & Kernel            |
+| ---------------------------------| ----------------------------------- |
+| Exclusive userns-remap (default) | Ubuntu 19.04 Disco (>= 5.0.0-21.22) |
+|                                  | Ubuntu 18.04 Bionic (5.0+ kernel)   |
+| Docker userns-remap              | Ubuntu 19.04 Disco                  |
+|                                  | Ubuntu 18.10 Cosmic                 |
+|                                  | Ubuntu 18.04 Bionic                 |
 
 See [here](docs/usage.md#system-container-isolation-modes) for info on
 Sysbox isolation modes and how to configure them.
@@ -211,16 +208,17 @@ sysbox-mgr.service                  loaded    active   running sysbox-mgr compon
 sysbox.service                     loaded    active   exited  Sysbox General Service
 ```
 
-Note: the sysbox.service is ephemeral (it exits once it launches the other sysbox services).
+Note: the sysbox.service is ephemeral (it exits once it launches the other sysbox services; that's why
+you see `sysbox.service   loaded  active  exited` above).
 
 If you are curious on what the other Sysbox services are, refer to the [Sysbox design document](docs/design.md).
 
 If you hit problems during installation, see the [Troubleshooting document](docs/troubleshoot.md).
 
 ## Usage
 
-To launch a system container with Docker, point Docker to the Sysbox container
-runtime, using the `--runtime=sysbox-runc` option:
+To launch a system container with Docker, simply point Docker to the
+Sysbox container runtime as follows:
 
 ```console
 $ docker run --runtime=sysbox-runc --rm -it --hostname my_cont debian:latest
@@ -303,9 +301,11 @@ We don't yet support other container managers (e.g., cri-o, etc).
 
 ## Production Readiness
 
-Sysbox is still in alpha / experimental stage. It's **not production ready yet**.
+Sysbox is well tested and will work for all use-cases described in our
+documentation.
 
-Nestybox is actively enhancing its functionality and fixing issues at this time.
+Having said that, it's early days for Nestybox so the product is not
+production ready yet.
 
 Your feedback regarding issues or improvements is much appreciated!
 
diff --git a/docs/troubleshoot.md b/docs/troubleshoot.md
@@ -6,14 +6,16 @@
     -   [Bionic Beaver](#bionic-beaver)
     -   [Disco Dingo](#disco-dingo)
 -   [Sysbox Installation Problems](#sysbox-installation-problems)
--   [Sysbox Logs](#sysbox-logs)
-    -   [sysbox-mgr and sysbox-fs](#sysbox-mgr-and-sysbox-fs)
-    -   [sysbox-runc](#sysbox-runc)
--   [Docker reports Unknown Runtime error](#docker-reports-unknown-runtime-error)
--   [Bind Mount Permissions Error](#bind-mount-permissions-error)
 -   [Ubuntu Shiftfs Module Not Present](#ubuntu-shiftfs-module-not-present)
 -   [Unprivileged User Namespace Creation Error](#unprivileged-user-namespace-creation-error)
+-   [Docker reports Unknown Runtime error](#docker-reports-unknown-runtime-error)
+-   [Bind Mount Permissions Error](#bind-mount-permissions-error)
 -   [Failed to Setup Docker Volume Manager Error](#failed-to-setup-docker-volume-manager-error)
+-   [Failed to stat mount source at /var/lib/sysboxfs](#failed-to-stat-mount-source-at-varlibsysboxfs)
+-   [Failed to register with sysbox-mgr](#failed-to-register-with-sysbox-mgr)
+-   [Sysbox Logs](#sysbox-logs)
+    -   [sysbox-mgr and sysbox-fs](#sysbox-mgr-and-sysbox-fs)
+    -   [sysbox-runc](#sysbox-runc)
 
 ## Upgrading the Ubuntu Kernel
 
@@ -106,29 +108,58 @@ sysbox.service                     loaded    active   exited  Sysbox General Ser
 The sysbox.service is ephemeral (it exits once it launches the other sysbox services),
 so the `active exited` status above is expected.
 
-## Sysbox Logs
+## Ubuntu Shiftfs Module Not Present
 
-### sysbox-mgr and sysbox-fs
+When creating a system container, the following error indicates that
+the Ubuntu shiftfs module is required by Sysbox but is not loaded
+in the Linux kernel:
 
-The Sysbox daemons (i.e. sysbox-fs and sysbox-mgr) will log
-information related to their activities in the
-`/var/log/sysbox-fs.log` and `/var/log/sysbox-mgr.log` files
-respectively. These logs should be useful during troubleshooting
-exercises.
+```console
+# docker run --runtime=sysbox-runc -it debian:latest
+docker: Error response from daemon: OCI runtime create failed: container requires user-ID shifting but error was found: shiftfs module is not loaded in the kernel. Update your kernel to include shiftfs module or enable Docker with userns-remap. Refer to the Sysbox troubleshooting guide for more info: unknown
+```
 
-### sysbox-runc
+The error likely means you are running Sysbox on an older Ubuntu
+kernel, as newer Ubuntu kernels come with shiftfs.
 
-For sysbox-runc, logging is handled as follows:
+The Ubuntu shiftfs module is required when Sysbox is configured in
+[exclusive userns-remap mode](usage.md#exclusive-userns-remap-mode)
+(it's default operating mode).
 
--   When running Docker + sysbox-runc, the sysbox-runc logs are actually stored in
-    a containerd directory such as:
+You can work-around this error by either:
 
-    `/run/containerd/io.containerd.runtime.v1.linux/moby/<container-id>/log.json`
+-   Updating your Linux distro. See [here](../README.md#supported-linux-distros)
+    for the list of Linux distros supported by Sysbox, and
+    [here](#upgrading-the-ubuntu-kernel) for recommendations on how to
+    update the distro.
 
-    where `<container-id>` is the container ID returned by Docker.
+or
 
--   When running sysbox-runc directly, sysbox-runc will not produce any logs by default.
-    Use the `sysbox-runc --log` option to change this.
+-   Configuring Sysbox in docker userns-remap mode, as described
+    [here](usage.md#system-container-isolation-modes). This
+    mode does not require use of shiftfs.
+
+## Unprivileged User Namespace Creation Error
+
+When creating a system container, Docker may report the following error:
+
+```console
+docker run --runtime=sysbox-runc -it ubuntu:latest
+docker: Error response from daemon: OCI runtime create failed: host is not configured properly: kernel is not configured to allow unprivileged users to create namespaces: /proc/sys/kernel/unprivileged_userns_clone: want 1, have 0: unknown.
+```
+
+This means that the host's kernel is not configured to allow unprivileged users
+to create user namespaces.
+
+For Ubuntu, fix this with:
+
+```console
+sudo sh -c "echo 1 > /proc/sys/kernel/unprivileged_userns_clone"
+```
+
+**Note:** The Sysbox package installer automatically executes this
+instruction, so normally there is no need to do this configuration
+manually.
 
 ## Docker reports Unknown Runtime error
 
@@ -180,77 +211,98 @@ system container to a non-root user on the host.
 Refer to [System Container Bind Mount Requirements](usage.md#system-container-bind-mount-requirements) for
 info on how to set the correct permissions on the bind mount.
 
-## Ubuntu Shiftfs Module Not Present
+## Failed to Setup Docker Volume Manager Error
 
-When creating a system container, the following error indicates that
-the Ubuntu shiftfs module is required by Sysbox but is not loaded
-in the Linux kernel:
+When creating a system container, Docker may report the following error:
 
 ```console
-# docker run --runtime=sysbox-runc -it debian:latest
-docker: Error response from daemon: OCI runtime create failed: container requires user-ID shifting but error was found: shiftfs module is not loaded in the kernel. Update your kernel to include shiftfs module or enable Docker with userns-remap. Refer to the Sysbox troubleshooting guide for more info: unknown
+docker run --runtime=sysbox-runc -it ubuntu:latest
+docker: Error response from daemon: OCI runtime create failed: failed to setup docker volume manager: host dir for docker store /var/lib/sysbox/docker can't be on ..."
 ```
 
-The error likely means you are running Sysbox on an older Ubuntu
-kernel, as newer Ubuntu kernel come with shiftfs.
+This means that Sysbox's `/var/lib/sysbox` directory is on a
+filesystem not supported by Sysbox.
 
-The Ubuntu shiftfs module is required when Sysbox is configured in
-[exclusive userns-remap mode](usage.md#exclusive-userns-remap-mode)
-(it's default operating mode).
+This directory must be on one of the following filesystems:
 
-You can work-around this error by either:
+-   ext4
+-   btrfs
 
--   Updating your Linux distro. See
-    [here](../README.md#supported-linux-distros) for the list of Linux
-    distros supported by Sysbox, and [here](#upgrading-the-ubuntu-kernel)
-    for recommendations on how to update the distro.
+The same requirement applies to the `/var/lib/docker` directory.
 
-or
+This is normally the case for vanilla Ubuntu installations, so this
+error is not common.
 
--   Configuring Sysbox in docker userns-remap mode, as described
-    [here](usage.md#system-container-isolation-modes). This
-    mode does not require use of shiftfs.
+## Failed to stat mount source at /var/lib/sysboxfs
 
-## Unprivileged User Namespace Creation Error
+While creating a system container, Docker may report the following error:
 
-When creating a system container, Docker may report the following error:
+```console
+$ docker run --runtime=sysbox-runc -it alpine
+docker: Error response from daemon: OCI runtime create failed: failed to create lib container mount: failed to stat mount source at /var/lib/sysboxfs/proc/sys: stat /var/lib/sysboxfs/proc/sys: no such file or directory: unknown.
+```
+
+This likely means that the sysbox-fs daemon is not running (for some reason).
+
+Check if the sysbox-fs process is running via `ps`. Ideally it should look like this:
 
 ```console
-docker run --runtime=sysbox-runc -it ubuntu:latest
-docker: Error response from daemon: OCI runtime create failed: host is not configured properly: kernel is not configured to allow unprivileged users to create namespaces: /proc/sys/kernel/unprivileged_userns_clone: want 1, have 0: unknown.
+$ ps -fu root | grep sysbox
+root     23945     1  0 Nov12 pts/0    00:00:00 sysbox-fs --log-level=debug --log /dev/stdout
+root     23946     1  0 Nov12 pts/0    00:00:00 sysbox-mgr --log-level=debug --log /dev/stdout
 ```
 
-This means that the host's kernel is not configured to allow unprivileged users
-to create user namespaces.
+If sysbox-fs is missing from the `ps` output, stop and restart Sysbox via Systemd:
 
-For Ubuntu, fix this with:
+```console
+$ sudo systemctl restart sysbox
+```
+
+## Failed to register with sysbox-mgr
+
+While creating a system container, Docker may report the following error:
 
 ```console
-sudo sh -c "echo 1 > /proc/sys/kernel/unprivileged_userns_clone"
+$ docker run --runtime=sysbox-runc -it alpine
+docker: Error response from daemon: OCI runtime create failed: failed to register with sysbox-mgr: failed to invoke Register via grpc: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: Error while dialing dial unix /run/sysbox/sysmgr.sock: connect: connection refused": unknown.
 ```
 
-**Note:** The Sysbox package installer automatically executes this
-instruction, so normally there is no need to do this configuration
-manually.
+This likely means that the sysbox-mgr daemon is not running (for some reason).
 
-## Failed to Setup Docker Volume Manager Error
+Check if the sysbox-mgr process is running via `ps`. Ideally it should look like this:
 
-When creating a system container, Docker may report the following error:
+```console
+$ ps -fu root | grep sysbox
+root     23945     1  0 Nov12 pts/0    00:00:00 sysbox-fs --log-level=debug --log /dev/stdout
+root     23946     1  0 Nov12 pts/0    00:00:00 sysbox-mgr --log-level=debug --log /dev/stdout
+```
+
+If sysbox-mgr is missing from the `ps` output, stop and restart Sysbox via Systemd:
 
 ```console
-docker run --runtime=sysbox-runc -it ubuntu:latest
-docker: Error response from daemon: OCI runtime create failed: failed to setup docker volume manager: host dir for docker store /var/lib/sysbox/docker can't be on ..."
+$ sudo systemctl restart sysbox
 ```
 
-This means that Sysbox's `/var/lib/sysbox` directory is on a
-filesystem not supported by Sysbox.
+## Sysbox Logs
 
-This directory must be on one of the following filesystems:
+### sysbox-mgr and sysbox-fs
 
--   ext4
--   btrfs
+The Sysbox daemons (i.e. sysbox-fs and sysbox-mgr) will log
+information related to their activities in the
+`/var/log/sysbox-fs.log` and `/var/log/sysbox-mgr.log` files
+respectively. These logs should be useful during troubleshooting
+exercises.
 
-The same requirement applies to the `/var/lib/docker` directory.
+### sysbox-runc
 
-This is normally the case for vanilla Ubuntu installations, so this
-error is not common.
+For sysbox-runc, logging is handled as follows:
+
+-   When running Docker + sysbox-runc, the sysbox-runc logs are actually stored in
+    a containerd directory such as:
+
+    `/run/containerd/io.containerd.runtime.v1.linux/moby/<container-id>/log.json`
+
+    where `<container-id>` is the container ID returned by Docker.
+
+-   When running sysbox-runc directly, sysbox-runc will not produce any logs by default.
+    Use the `sysbox-runc --log` option to change this.
