|
17 | 17 | ~ under the License. |
18 | 18 | --> |
19 | 19 |
|
20 | | -# Using Podman instead of Docker |
| 20 | +# Container Runtimes |
21 | 21 |
|
22 | | -Iceberg-rust does not require containerization, except for integration tests, where "docker" and "docker-compose" are used to start containers for minio and various catalogs. Below instructions setup "rootful podman" and docker's official docker-compose plugin to run integration tests as an alternative to docker or Orbstack. |
| 22 | +Iceberg-rust uses containers for integration tests, where `docker` and `docker compose` start containers for MinIO and various catalogs. You can use any of the following container runtimes. |
| 23 | + |
| 24 | +## Docker Desktop |
| 25 | + |
| 26 | +[Docker Desktop](https://www.docker.com/products/docker-desktop/) is available for macOS, Windows, and Linux. |
| 27 | + |
| 28 | +1. Install Docker Desktop by downloading the [installer](https://www.docker.com/products/docker-desktop/) or using Homebrew on macOS. |
| 29 | + ```shell |
| 30 | + brew install --cask docker |
| 31 | + ``` |
| 32 | + |
| 33 | +2. Launch Docker Desktop and complete the setup. |
| 34 | + |
| 35 | +3. Verify the installation. |
| 36 | + ```shell |
| 37 | + docker --version |
| 38 | + docker compose version |
| 39 | + ``` |
| 40 | + |
| 41 | +4. Try some integration tests! |
| 42 | + ```shell |
| 43 | + make test |
| 44 | + ``` |
| 45 | + |
| 46 | +## OrbStack (macOS) |
| 47 | + |
| 48 | +[OrbStack](https://orbstack.dev/) is a lightweight alternative to Docker Desktop on macOS. |
| 49 | + |
| 50 | +1. Install OrbStack by downloading the [installer](https://orbstack.dev/download) or using Homebrew. |
| 51 | + ```shell |
| 52 | + brew install orbstack |
| 53 | + ``` |
| 54 | + |
| 55 | +2. Migrate Docker data (if switching from Docker Desktop). |
| 56 | + ```shell |
| 57 | + orb migrate docker |
| 58 | + ``` |
| 59 | + |
| 60 | +3. (Optional) Add registry mirrors. |
| 61 | + |
| 62 | + You can edit the config directly at `~/.orbstack/config/docker.json` and restart the engine with `orb restart docker`. |
| 63 | + |
| 64 | + ```json |
| 65 | + { |
| 66 | + "registry-mirrors": ["<mirror_addr>"] |
| 67 | + } |
| 68 | + ``` |
| 69 | + |
| 70 | +4. Try some integration tests! |
| 71 | + ```shell |
| 72 | + make test |
| 73 | + ``` |
| 74 | + |
| 75 | +## Podman |
| 76 | + |
| 77 | +[Podman](https://podman.io/) is a daemonless container engine. The instructions below set up "rootful podman" with docker's official docker-compose plugin. |
23 | 78 |
|
24 | 79 | 1. Have podman v4 or newer. |
25 | 80 | ```shell |
26 | 81 | $ podman --version |
27 | 82 | podman version 4.9.4-rhel |
28 | 83 | ``` |
| 84 | +
|
29 | 85 | 2. Create a docker wrapper script: |
30 | 86 | |
31 | | - * Create a fresh `/usr/bin/docker` file and add the below contents: |
| 87 | + Create a fresh `/usr/bin/docker` file and add the below contents: |
32 | 88 | ```bash |
33 | 89 | #!/bin/sh |
34 | 90 | [ -e /etc/containers/nodocker ] || \ |
35 | 91 | echo "Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg." >&2 |
36 | 92 | exec sudo /usr/bin/podman "$@" |
37 | 93 | ``` |
38 | 94 |
|
39 | | - * Set new `/usr/bin/docker` file to executable. |
| 95 | + Set new `/usr/bin/docker` file to executable. |
40 | 96 | ```shell |
41 | 97 | sudo chmod +x /usr/bin/docker |
42 | 98 | ``` |
@@ -78,23 +134,24 @@ Iceberg-rust does not require containerization, except for integration tests, wh |
78 | 134 | cargo test -p iceberg --test file_io_s3_test |
79 | 135 | ``` |
80 | 136 |
|
81 | | -# References |
| 137 | +### Note on rootless containers |
82 | 138 |
|
83 | | -* <https://docs.docker.com/compose/install/linux> |
84 | | -* <https://www.redhat.com/sysadmin/podman-docker-compose> |
| 139 | +As of podman v4, ["To be succinct and simple, when running rootless containers, the container itself does not have an IP address"](https://www.redhat.com/sysadmin/container-ip-address-podman). This causes issues with iceberg-rust's integration tests, which rely upon IP-addressable containers via docker-compose. As a result, podman "rootful" containers are required to ensure containers have IP addresses. |
| 140 | + |
| 141 | +### Podman troubleshooting |
| 142 | + |
| 143 | +**Error:** `short-name "apache/iceberg-rest-fixture" did not resolve to an alias and no unqualified-search registries are defined in "/etc/containers/registries.conf"` |
85 | 144 |
|
86 | | -# Note on rootless containers |
| 145 | +**Fix:** Add or modify the `/etc/containers/registries.conf` file: |
| 146 | +```toml |
| 147 | +[[registry]] |
| 148 | +prefix = "docker.io" |
| 149 | +location = "docker.io" |
| 150 | +``` |
87 | 151 |
|
88 | | -As of podman v4, ["To be succinct and simple, when running rootless containers, the container itself does not have an IP address"](https://www.redhat.com/sysadmin/container-ip-address-podman) This causes issues with iceberg-rust's integration tests, which rely upon ip-addressable containers via docker-compose. As a result, podman "rootful" containers are required throughout to ensure containers have IP addresses. Perhaps as a future work or with updates to default podman networking, the need for "rootful" podman containers can be eliminated. |
| 152 | +### Podman references |
89 | 153 |
|
| 154 | +* <https://docs.docker.com/compose/install/linux> |
| 155 | +* <https://www.redhat.com/sysadmin/podman-docker-compose> |
90 | 156 | * <https://www.redhat.com/sysadmin/container-ip-address-podman> |
91 | 157 | * <https://github.com/containers/podman/blob/main/docs/tutorials/basic_networking.md> |
92 | | -
|
93 | | -# Debugging Note: |
94 | | - - Fix for error: `Error: short-name "apache/iceberg-rest-fixture" did not resolve to an alias and no unqualified-search registries are defined in "/etc/containers/registries.conf"` |
95 | | - - Add or modify the `/etc/containers/registries.conf` file: |
96 | | - ```toml |
97 | | - [[registry]] |
98 | | - prefix = "docker.io" |
99 | | - location = "docker.io" |
100 | | - ``` |
|
0 commit comments