Add --force-rm flag (#9727)

* feat: add `--force-rm` flag to configure removing intermediate docker layers

This commit makes it possible to turn off the removal of intermediate
docker layers when building the tfb containers by adding a `--force-rm`
flag to the tfb script. This is useful in situations where you want to
inspect the intermediate layers for debugging purposes, or for caching
builds of dependencies as a docker layer to speed up the build process.

Note that the default behavior is to remove the intermediate layers to
avoid filling up the disk with unused layers on the citrine server.

Fixes: #9718

* hyper: cache dependencies to reduce build time

This change reduces the time it takes to build the hyper Docker image by
caching the dependency builds. This is done by building a dummy binary
before copying the source code into the image.

* Change default for --force-rm to False

---------

Co-authored-by: Mike Smith <[email protected]>

Author: joshka

frameworks/Rust/hyper/hyper.dockerfile

@@ -1,8 +1,17 @@
 FROM rust:1.85 AS hyper
 
 WORKDIR /src
-COPY . .
-RUN RUSTFLAGS="-C target-cpu=native" cargo install --path . --locked
+ENV RUSTFLAGS="-C target-cpu=native"
+
+# Cache dependency builds (requires passing --force-rm False to tfb command)
+COPY Cargo.toml Cargo.lock /src/
+RUN mkdir src \
+    && echo "fn main() {println!(\"if you see this, the build broke\")}" > src/main.rs \
+    && cargo build --release \
+    && rm -rfv src/ target/release/hyper-techempower* target/release/deps/hyper_techempower*
+
+COPY . /src/
+RUN cargo install --path . --locked
 EXPOSE 8080
 CMD ["hyper-techempower"]
 HEALTHCHECK CMD curl --fail http://localhost:8080/ping || exit 1

toolset/run-tests.py

@@ -208,6 +208,10 @@ def main(argv=None):
         nargs='*',
         default=None,
         help='Extra docker arguments to be passed to the test container')
+    parser.add_argument(
+        '--force-rm',
+        default=False,
+        help='Remove intermediate docker containers after running.')
 
     # Network options
     parser.add_argument(

toolset/utils/benchmark_config.py

@@ -57,6 +57,7 @@ def __init__(self, args):
         self.cpuset_cpus = args.cpuset_cpus
         self.test_container_memory = args.test_container_memory
         self.extra_docker_runtime_args = args.extra_docker_runtime_args
+        self.force_rm_intermediate_docker_layers = args.force_rm
 
         if self.network_mode is None:
             self.network = 'tfb'

toolset/utils/docker_helper.py

@@ -39,7 +39,7 @@ def __build(self, base_url, path, build_log_file, log_prefix, dockerfile,
                     path=path,
                     dockerfile=dockerfile,
                     tag=tag,
-                    forcerm=True,
+                    forcerm=self.benchmarker.config.force_rm_intermediate_docker_layers,
                     timeout=3600,
                     pull=True,
                     buildargs=buildargs,