Spike: Use hardened images

[tushortz]

Description

This PR includes the following:

• docker image hardening modification

Fixes #CDD-3109

---

Type of change

Please select the options that are relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Tech debt item (this is focused solely on addressing any relevant technical debt)

---

Checklist:

• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests at the right levels to prove my change is effective
• I have added screenshots or screen grabs where appropriate
• I have added docstrings in the correct style (google)

[tushortz]

SonarCloud complaints doesn't have a solution at the moment because the available docker hardening images don't provide rootless image unless we go for the enterprise version

[tushortz]

We may need an account for dhi.io if we don't have one yet

[jrdh]

I'll go through and do a review of the individual files but a couple of notable things so far that I've found that I'll wait on before I carry on:

Build errors

I can't get the build to work 🙁 If I run docker build -f Dockerfile -t dd-api . it builds but spits out a warning:

 1 warning found (use docker --debug to expand):
 - InvalidBaseImagePlatform: Base image gcr.io/distroless/base-debian12:nonroot-arm64 was pulled with platform "linux/arm64", expected "linux/amd64" for current build (line 33)

You get a warning when attempting to run the image too.
I assume these are because the base debian image is dependant on the platform it's built on whereas the prod stage is specifically pinned to arm64 (the image tag has it in it) so without specifying the build platform you can get mismatches (e.g. on my machine which is amd64 i get this warning presumably because the debian image is amd64 whereas the prod stage is based on an arm64 image).

If I run docker build --platform linux/arm64 -f Dockerfile -t dd-api . it doesn't:

 => ERROR [build 6/7] RUN bash /usr/local/bin/build_distroless_runtime.sh                                                                                                                                                                0.1s
------                                                                                                                                                                                                                                        
 > [build 6/7] RUN bash /usr/local/bin/build_distroless_runtime.sh:
0.086 exec /bin/sh: exec format error
------
Dockerfile:25
--------------------
  23 |     # Dockerfile stays readable and the build logic can be tested/iterated on.
  24 |     COPY docker/build_distroless_runtime.sh /usr/local/bin/build_distroless_runtime.sh
  25 | >>> RUN bash /usr/local/bin/build_distroless_runtime.sh
  26 |     
  27 |     # Mounts the application code into the image
--------------------
ERROR: failed to build: failed to solve: process "/bin/sh -c bash /usr/local/bin/build_distroless_runtime.sh" did not complete successfully: exit code: 255

distroless & the shell

The point of a distroless image is that it only has your application runtime in it, from the docs for the image you're using:

"Distroless" images contain only your application and its runtime dependencies. They do not contain package managers, shells or any other programs you would expect to find in a standard Linux distribution.

To this end, I don't think we should be copying bash into the distroless image as this defeats the point. The problem with this is that we use bash* to run random tasks through a utility worker which invokes the uhd script directly, e.g. this bootstrap task.

So I think we have a choice - we either accept that a distroless image we've copied bash* into is better than a base python image with apt etc in it, or we don't bother doing this until we can remove the dependency on a shell script for management tasks.

I'm inclined to think we're unlikely to get rid of the shell scripts any time soon (e.g. by converting it to python) so we should probably just do as you have done and copy a shell into the distroless image. Appreciate that this comment doesn't actually ask you to change anything but it's useful as a historical context of why this is how it is.

*we don't actually use bash, we use zsh for these shell scripts. E.g. all the github actions install zsh before using uhd, so we should probably use that instead. You might even be able to get away with installing a statically linked version and therefore not having to copy all the libs, e.g. https://packages.debian.org/bookworm/zsh-static. Worth a try at least!

python libs

Do we need to copy all the python libs in the way you have? Referencing Sam's (one of our devops engineers who had a play with this) work he just copies a few specific dirs (https://github.com/ukhsa-collaboration/distroless-base/blob/main/python3/Dockerfile#L17). Does his solution copy more than necessary for example? The only reason I bring it up is because it's much simpler than your shell script and therefore easier to reason about and explain.

[sonarqubecloud]

Quality Gate failed

Failed conditions
3 Security Hotspots

See analysis details on SonarQube Cloud