|
| 1 | +# Build Ceph within a Container |
| 2 | + |
| 3 | +The Ceph project includes a script and additional files that help build |
| 4 | +the ceph.git sources inside an OCI-style container. This script requires |
| 5 | +Python 3 (tested with 3.8 or later) and Podman or Docker. |
| 6 | + |
| 7 | +The script aims to make it simple for a developer or anyone |
| 8 | +wanting to compile Ceph on Linux but not wanting to dedicate a full |
| 9 | +physical or virtual host to the job. The containers encapsulate the |
| 10 | +build dependencies and environment and are quick and easy to clean |
| 11 | +up when you no longer need them. |
| 12 | + |
| 13 | +# build-with-containers.py Introduction |
| 14 | + |
| 15 | +The script exists within the ceph.git tree at |
| 16 | +`src/script/build-with-container.py`. The script operates on the entire Ceph |
| 17 | +repository and will automatically try to detect the root of the ceph.git tree. |
| 18 | +At any time the `--help` option can be passed to the script for a complete |
| 19 | +listing of command line options. |
| 20 | + |
| 21 | +To start with the default options run `./src/script/build-with-container.py`. |
| 22 | +The script will first check if a build image already exists, if not it |
| 23 | +will construct one using the `Dockerfile.build` file in the Ceph tree [1]. |
| 24 | + |
| 25 | +Once a build container is available the script will build Ceph from source using |
| 26 | +a build directory named `build` in the root of the source tree. The default |
| 27 | +environment uses a base distribution of `centos9`. |
| 28 | + |
| 29 | +You can select the base distribution to use with the `--distro`/`-d` option. |
| 30 | +The list of available distribution bases include `ubuntu22.04` and `centos9`. |
| 31 | +These choices are known to work well. Other more experimental choices, |
| 32 | +including `ubuntu24.04` and `centos10`, are available as well but do not expect |
| 33 | +these platforms to work without tinkering. This option is a shorthand for |
| 34 | +specifying the base container image that will be used to construct the build |
| 35 | +image, the tags used for that image, as well as helping determine what kind of |
| 36 | +packages can be produced using that build image. |
| 37 | + |
| 38 | +You can specify where build artifacts are written with the `--build-dir`/`-b` |
| 39 | +option. For example `./src/script/build-with-container.py -b build.try1`. |
| 40 | + |
| 41 | +The tool supports mutliple build targets. Often these targets are chained |
| 42 | +together - for example almost all targets depend on the container target. To |
| 43 | +select a target use the `--execute`/`-e` command line option. For example: |
| 44 | +`./src/script/build-with-container.py -e tests` will execute the unit tests |
| 45 | +after building them. (Note: the `--no-prereqs` option exists to disable the |
| 46 | +chaining but this should not be needed in most circumstances) |
| 47 | + |
| 48 | + |
| 49 | +[1] - This behavior can be customized with the `--image-sources`/`-I` |
| 50 | +option. |
| 51 | + |
| 52 | + |
| 53 | +## Examples: |
| 54 | + |
| 55 | +Build from source code on CentOS 9, with a build directory named for the base |
| 56 | +distribution: |
| 57 | +``` |
| 58 | +./src/script/build-with-container.py -d centos9 -b build.centos9 -e build |
| 59 | +``` |
| 60 | + |
| 61 | +Build from source code on Ubuntu 22.04, with a build directory named for the |
| 62 | +base distribution: |
| 63 | +``` |
| 64 | +./src/script/build-with-container.py -d ubuntu22.04 -b build.u2204 -e build |
| 65 | +``` |
| 66 | + |
| 67 | +Build RPM packages on Centos 9: |
| 68 | +``` |
| 69 | +./src/script/build-with-container.py -d centos9 -e packages |
| 70 | +``` |
| 71 | + |
| 72 | +Build Debian packages on Ubuntu 22.04 (Jammy): |
| 73 | +``` |
| 74 | +./src/script/build-with-container.py -d ubuntu22.04 -e packages |
| 75 | +``` |
| 76 | + |
| 77 | +## Common Targets |
| 78 | +* build - Build Ceph sources. Compiles C/C++ code, etc |
| 79 | +* tests - Execute unit tests. Runs the unit test suite, depends on `buildtests` to compile some of the test suites |
| 80 | +* custom - Execute a custom command. See description below |
| 81 | +* packages - Build Ceph packages of the selected distribution's native package type |
| 82 | +* interactive - Start the build container in an interactive mode |
| 83 | + |
| 84 | +### Custom Commands |
| 85 | + |
| 86 | +The `custom` target can be used to run a single command that the script is not |
| 87 | +already programmed to handle. The custom command must come after a `--` to terminate |
| 88 | +the normal command line arguments for `build-with-container.py`. For example: |
| 89 | +``` |
| 90 | +./src/script/build-with-container.py -d ubuntu22.04 -e custom -- shellcheck src/script/buildcontainer-setup.sh |
| 91 | +``` |
| 92 | + |
| 93 | +### Interactive mode |
| 94 | + |
| 95 | +The `interactive` target can be used to run a shell within the container. This |
| 96 | +is handy for those times you want to run multiple commands, by hand, within the |
| 97 | +container environment. |
| 98 | +As an example: |
| 99 | +``` |
| 100 | +./src/script/build-with-container.py -d ubuntu22.04 -e interactive |
| 101 | +``` |
| 102 | + |
| 103 | + |
| 104 | +## Additional Features |
| 105 | + |
| 106 | +### Control build image name and tags |
| 107 | + |
| 108 | +The `build-with-container.py` script automatically generates images names based |
| 109 | +on a standard name and auto-generated tag. By default the script names the |
| 110 | +image `ceph-build` and assumes the images are local only - the image name will |
| 111 | +not refer to any image registry. The images are tagged with the name of the |
| 112 | +current branch and base distribution. For example, assuming we're on a branch |
| 113 | +named `wip-test` and we execute the script with `-d ubuntu22.04` we will expect |
| 114 | +or build an image named `ceph-build:wip-test.ubuntu22.04`. |
| 115 | + |
| 116 | +The image name/repository can be customized using the `--image-repo` option. |
| 117 | +The tag can be overridden by the `--tag` option or extended by using the |
| 118 | +`--tag` option with a plus (+) character at the start of the value. For |
| 119 | +example: `./src/script/build-with-container.py |
| 120 | +--image-repo=quay.io/example/build-example --tag=foobar` would create or reuse |
| 121 | +an image named `quay.io/example/build-example:foobar`. |
| 122 | +`./src/script/build-with-container.py |
| 123 | +--image-repo=quay.io/example/build-example --distro=centos9 --tag=+foobar` |
| 124 | +would use an image named |
| 125 | +`quay.io/example/build-example:wip-test.centos9.foobar`. |
| 126 | + |
| 127 | +If one wants to override the name of the branch or the branch can not be |
| 128 | +automatically detected the `--current-branch` option can be supplied to |
| 129 | +customize this value. |
| 130 | + |
| 131 | + |
| 132 | +### Control build image source |
| 133 | + |
| 134 | +By default `build-with-container.py` will try reuse build images if they are |
| 135 | +cached in the local container store. If the image is not present it will build |
| 136 | +one. In addition to these default actions the script can be intructed to "pull" |
| 137 | +an image from a remote registry, possibly avoiding the need to build an image. |
| 138 | + |
| 139 | +How the build image is acquired can be controlled using the `--image-sources` |
| 140 | +option. The option takes a comma-separated list of terms. The terms are |
| 141 | +`cache`, `pull`, and `build`: |
| 142 | +* build - Create a new build image |
| 143 | +* cache - Check for an existing image in local container storage |
| 144 | +* pull - Pull an image form a container image registry |
| 145 | + |
| 146 | +So for example if you did not want to fall back to building an image locally, |
| 147 | +you could pass `--image-sources=cache,pull` to the script. Passing |
| 148 | +`--image-sources=build` will force the script to rebuild an image even if would |
| 149 | +be available elsewhere. |
| 150 | + |
| 151 | +When an image is to be built the image base is typically derived from the name |
| 152 | +of the distribution being used. However, the base image can be overridden on |
| 153 | +the command line using the `--base-image` option. For example, if one had a |
| 154 | +local registry with a CentOS 9 (Stream) base image the following example could |
| 155 | +be used: `./src/script/build-with-container.py -d centos9 --base-image |
| 156 | +myreg.example.com/ceph/centos-base:9` |
| 157 | + |
| 158 | + |
| 159 | +### Controlling where files are written |
| 160 | + |
| 161 | +By default the directory holding the Ceph source tree is mounted at `/ceph` |
| 162 | +within the container (controlled by the `--homedir` option). Various build tasks |
| 163 | +will write files to this directory. In some cases it's useful to keep the |
| 164 | +directory free from changes and so the `--overlay-dir` option can be used to |
| 165 | +make that volume use an overlay. |
| 166 | + |
| 167 | +The overlay directory will be automatically created if needed and will contain |
| 168 | +a `content` directory for new or updated files and a `work` directory, a |
| 169 | +special directory needed by the overlayfs. For example: |
| 170 | +`./src/script/build-with-container.py --overlay-dir build.ovr -b build.inner -d |
| 171 | +centos9` will end up creating a directory `build.ovr/content/build.inner` which |
| 172 | +will contain the results of the compile that would typically appear in just |
| 173 | +`build.inner`. Other writes that would have normally effected the source tree |
| 174 | +will appear in `build.ovr/content` |
| 175 | + |
| 176 | +The overlay can also be temporary, with no files persisted after the container |
| 177 | +has exited. Pass `--overlay-dir=-` to enable this option. Note that invoking |
| 178 | +`build-with-container.py` default targets may use multiple container instances |
| 179 | +and passing this option will break those targets. |
0 commit comments