Refactor shell scripts (nhs-england-tools#137)

## Description

This is the first part of the refactoring effort to ensure that the
shell scripts:

- Follow a consistent convention,
- Pass the ShellCheck linting test (please run `make
shellscript-lint-all`), and
- Work either with the pre-installed CLI tooling or run commands using
Docker containers.

## Context

Information for a reviewer:

- The first two commits,
[a4517db](nhs-england-tools@a4517db)
and
[7b94f14](nhs-england-tools@7b94f14),
implement stylistic and linting recommendations.
- The last two commits,
[f8b9e84](nhs-england-tools@f8b9e84)
and
[22f7700](nhs-england-tools@22f7700),
refactor how tools like Gitleaks and SonarScanner are executed.

## Type of changes

- [x] Refactoring (non-breaking change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would change existing
functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)

## Checklist

- [x] I am familiar with the [contributing
guidelines](../docs/CONTRIBUTING.md)
- [x] I have followed the code style of the project
- [ ] I have added tests to cover my changes
- [ ] I have updated the documentation accordingly
- [ ] This PR is a result of pair or mob programming

---

## Sensitive Information Declaration

To ensure the utmost confidentiality and protect your and others
privacy, we kindly ask you to NOT including [PII (Personal Identifiable
Information) / PID (Personal Identifiable
Data)](https://digital.nhs.uk/data-and-information/keeping-data-safe-and-benefitting-the-public)
or any other sensitive data in this PR (Pull Request) and the codebase
changes. We will remove any PR that do contain any sensitive
information. We really appreciate your cooperation in this matter.

- [x] I confirm that neither PII/PID nor sensitive data are included in
this PR and the codebase changes.

---------

Co-authored-by: Alex Young <[email protected]>

Author: stefaniuk

.github/actions/check-markdown-format/action.yaml

@@ -7,4 +7,4 @@ runs:
       shell: bash
       run: |
         export BRANCH_NAME=origin/${{ github.event.repository.default_branch }}
-        ./scripts/githooks/check-markdown-format.sh
+        check=branch ./scripts/githooks/check-markdown-format.sh

.github/actions/scan-secrets/action.yaml

@@ -6,5 +6,5 @@ runs:
     - name: "Scan secrets"
       shell: bash
       run: |
-        export ALL_FILES=true # Do not change this line, as new patterns may be added or history may be rewritten
-        ./scripts/githooks/scan-secrets.sh
+        # Please do not change this `check=whole-history` setting, as new patterns may be added or history may be rewritten.
+        check=whole-history ./scripts/githooks/scan-secrets.sh

.tool-versions

@@ -6,14 +6,10 @@ pre-commit 3.4.0
 # ==============================================================================
 # The section below is reserved for Docker image versions.
 
-# terraform, SEE: https://hub.docker.com/r/hashicorp/terraform/tags
-# docker/hashicorp/terraform 1.5.6@sha256:180a7efa983386a27b43657ed610e9deed9e6c3848d54f9ea9b6cb8a5c8c25f5
-
-# shellcheck, SEE: https://hub.docker.com/r/koalaman/shellcheck/tags
-# docker/koalaman/shellcheck latest@sha256:e40388688bae0fcffdddb7e4dea49b900c18933b452add0930654b2dea3e7d5c
-
-# hadolint, SEE: https://hub.docker.com/r/hadolint/hadolint/tags
-# docker/hadolint/hadolint 2.12.0-alpine@sha256:7dba9a9f1a0350f6d021fb2f6f88900998a4fb0aaf8e4330aa8c38544f04db42
-
-# ghcr.io/nhs-england-tools/github-runner-image, SEE: https://github.com/nhs-england-tools/github-runner-image/pkgs/container/github-runner-image
-# docker/ghcr.io/nhs-england-tools/github-runner-image 20230909-321fd1e-rt@sha256:ce4fd6035dc450a50d3cbafb4986d60e77cb49a71ab60a053bb1b9518139a646
+# TODO: Move this section - consider using a different file for the repository template dependencies.
+# docker/ghcr.io/gitleaks/gitleaks v8.18.0@sha256:fd2b5cab12b563d2cc538b14631764a1c25577780e3b7dba71657d58da45d9d9 # SEE: https://github.com/gitleaks/gitleaks/pkgs/container/gitleaks
+# docker/ghcr.io/nhs-england-tools/github-runner-image 20230909-321fd1e-rt@sha256:ce4fd6035dc450a50d3cbafb4986d60e77cb49a71ab60a053bb1b9518139a646 # SEE: https://github.com/nhs-england-tools/github-runner-image/pkgs/container/github-runner-image
+# docker/hadolint/hadolint 2.12.0-alpine@sha256:7dba9a9f1a0350f6d021fb2f6f88900998a4fb0aaf8e4330aa8c38544f04db42 # SEE: https://hub.docker.com/r/hadolint/hadolint/tags
+# docker/hashicorp/terraform 1.5.6@sha256:180a7efa983386a27b43657ed610e9deed9e6c3848d54f9ea9b6cb8a5c8c25f5 # SEE: https://hub.docker.com/r/hashicorp/terraform/tags
+# docker/koalaman/shellcheck latest@sha256:e40388688bae0fcffdddb7e4dea49b900c18933b452add0930654b2dea3e7d5c # SEE: https://hub.docker.com/r/koalaman/shellcheck/tags
+# docker/sonarsource/sonar-scanner-cli 5.0.1@sha256:494ecc3b5b1ee1625bd377b3905c4284e4f0cc155cff397805a244dee1c7d575 # SEE: https://hub.docker.com/r/sonarsource/sonar-scanner-cli/tags

README.md

@@ -48,7 +48,8 @@ The following software packages, or their equivalents, are expected to be instal
 - [docker](https://www.docker.com/) container runtime or a compatible tool, e.g. [podman](https://podman.io/),
 - [asdf](https://asdf-vm.com/) version manager,
 - [GNU make](https://www.gnu.org/software/make/) 3.82 or later,
-- [GNU coreutils](https://www.gnu.org/software/coreutils/) and [GNU binutils](https://www.gnu.org/software/binutils/) may be required to build dependencies like Python, which may need to be compiled during installation. For macOS users, this has been scripted and automated by the `dotfiles` project; please see this [script](https://github.com/nhs-england-tools/dotfiles/blob/main/assets/20-install-base-packages.macos.sh) for details.
+- [GNU coreutils](https://www.gnu.org/software/coreutils/) and [GNU binutils](https://www.gnu.org/software/binutils/) may be required to build dependencies like Python, which may need to be compiled during installation. For macOS users, this has been scripted and automated by the `dotfiles` project; please see this [script](https://github.com/nhs-england-tools/dotfiles/blob/main/assets/20-install-base-packages.macos.sh) for details,
+- [jq](https://jqlang.github.io/jq/) a lightweight and flexible command-line JSON processor.
 
 > [!NOTE]<br>
 > The version of GNU make available by default on macOS is earlier than 3.82. You will need to upgrade it or certain `make` tasks will fail. On macOS, you will need [homebrew](https://brew.sh/) installed, then to install `make`, like so:

docs/adr/assets/ADR-003/examples/bash/script.sh

@@ -17,7 +17,7 @@ function main() {
 function get-jwt-token() {
 
   header=$(echo -n '{"alg":"RS256","typ":"JWT"}' | base64 | tr -d '=' | tr -d '\n=' | tr -- '+/' '-_')
-  payload=$(echo -n '{"iat":'$(date +%s)',"exp":'$(($(date +%s)+600))',"iss":"'$GITHUB_APP_ID'"}' | base64 | tr -d '\n=' | tr -- '+/' '-_')
+  payload=$(echo -n '{"iat":'"$(date +%s)"',"exp":'$(($(date +%s)+600))',"iss":"'"$GITHUB_APP_ID"'"}' | base64 | tr -d '\n=' | tr -- '+/' '-_')
   signature=$(echo -n "$header.$payload" | openssl dgst -binary -sha256 -sign "$GITHUB_APP_PK_FILE" | openssl base64 | tr -d '\n=' | tr -- '+/' '-_')
 
   echo "$header.$payload.$signature"
@@ -30,17 +30,17 @@ function get-installation-id() {
     -H "Accept: application/vnd.github.v3+json" \
     https://api.github.com/app/installations)
 
-  echo "$(echo $installations_response | jq '.[] | select(.account.login == "'"$GITHUB_ORG"'") .id')"
+  echo "$installations_response" | jq '.[] | select(.account.login == "'"$GITHUB_ORG"'") .id'
 }
 
 function get-access-token() {
 
   token_response=$(curl -sX POST \
     -H "Authorization: Bearer $jwt_token" \
     -H "Accept: application/vnd.github.v3+json" \
-    https://api.github.com/app/installations/$installation_id/access_tokens)
+    "https://api.github.com/app/installations/$installation_id/access_tokens")
 
-  echo "$(echo $token_response | jq .token -r)"
+  echo "$token_response" | jq .token -r
 }
 
 main

scripts/config/pre-commit.yaml

@@ -4,6 +4,7 @@ repos:
   - id: scan-secrets
     name: Scan Secrets
     entry: ./scripts/githooks/scan-secrets.sh
+    args: ["check=staged-changes"]
     language: script
     pass_filenames: false
 - repo: local
@@ -19,6 +20,7 @@ repos:
   - id: check-markdown-format
     name: Check Markdown Format
     entry: ./scripts/githooks/check-markdown-format.sh
+    args: ["check=staged-changes"]
     language: script
     pass_filenames: false
 - repo: local

scripts/docker/docker.lib.sh

@@ -168,7 +168,7 @@ function docker-get-image-version-and-pull() {
   local versions_file="${TOOL_VERSIONS:=$(git rev-parse --show-toplevel)/.tool-versions}"
   local version="latest"
   if [ -f "$versions_file" ]; then
-    line=$(grep "docker/${name} " "$versions_file" | sed "s/^#\s*//; s/\s*#.*$//" | grep "${match_version:-'.*'}")
+    line=$(grep "docker/${name} " "$versions_file" | sed "s/^#\s*//; s/\s*#.*$//" | grep "${match_version:-".*"}")
     [ -n "$line" ] && version=$(echo "$line" | awk '{print $2}')
   fi
 
@@ -177,7 +177,7 @@ function docker-get-image-version-and-pull() {
   local digest="$(echo "$version" | sed 's/^.*@//')"
 
   # Check if the image exists locally already
-  if ! docker images | awk '{ print $1 ":" $2 }' | grep "^${name}:${tag}$"; then
+  if ! docker images | awk '{ print $1 ":" $2 }' | grep -q "^${name}:${tag}$"; then
     if [ "$digest" != "latest" ]; then
       # Pull image by the digest sha256 and tag it
       docker pull \

scripts/docker/dockerfile-linter.sh

@@ -46,7 +46,8 @@ function docker-run-hadolint() {
   # shellcheck disable=SC1091
   source ./scripts/docker/docker.lib.sh
 
-  image=$(name=hadolint/hadolint docker-get-image-version-and-pull)
+  # shellcheck disable=SC2155
+  local image=$(name=hadolint/hadolint docker-get-image-version-and-pull)
   # shellcheck disable=SC2001
   docker run --rm --platform linux/amd64 \
     --volume "$PWD:/workdir" \

scripts/docker/tests/docker.test.sh

@@ -144,7 +144,7 @@ function test-docker-get-image-version-and-pull() {
 
 # ==============================================================================
 
-function is_arg_true() {
+function is-arg-true() {
 
   if [[ "$1" =~ ^(true|yes|y|on|1|TRUE|YES|Y|ON)$ ]]; then
     return 0
@@ -155,7 +155,7 @@ function is_arg_true() {
 
 # ==============================================================================
 
-is_arg_true "${VERBOSE:-false}" && set -x
+is-arg-true "${VERBOSE:-false}" && set -x
 
 main "$@"
 

scripts/githooks/check-file-format.sh

@@ -1,6 +1,8 @@
 #!/bin/bash
 
-set +e
+# WARNING: Please, DO NOT edit this file! It is maintained in the Repository Template (https://github.com/nhs-england-tools/repository-template). Raise a PR instead.
+
+set -euo pipefail
 
 # Pre-commit git hook to check the EditorConfig rules compliance over changed
 # files. It ensures all non-binary files across the codebase are formatted
@@ -17,7 +19,6 @@ set +e
 #   0 - All files are formatted correctly
 #   1 - Files are not formatted correctly
 #
-#
 # The `check` parameter controls which files are checked, so you can
 # limit the scope of the check according to what is appropriate at the
 # point the check is being applied.
@@ -45,9 +46,10 @@ image_version=2.7.1@sha256:dd3ca9ea50ef4518efe9be018d669ef9cf937f6bb5cfe2ef84ff2
 
 function main() {
 
-  cd $(git rev-parse --show-toplevel)
+  cd "$(git rev-parse --show-toplevel)"
 
-  is-arg-true "$dry_run" && dry_run_opt="--dry-run"
+  # shellcheck disable=SC2154
+  is-arg-true "${dry_run:-false}" && dry_run_opt="--dry-run"
 
   check=${check:-working-tree-changes}
   case $check in
@@ -68,16 +70,17 @@ function main() {
       ;;
   esac
 
-
   # We use /dev/null here as a backstop in case there are no files in the state
-  # we choose.  If the filter comes back empty, adding `/dev/null` onto it has
+  # we choose. If the filter comes back empty, adding `/dev/null` onto it has
   # the effect of preventing `ec` from treating "no files" as "all the files".
   docker run --rm --platform linux/amd64 \
-    --volume=$PWD:/check \
+    --volume "$PWD":/check \
     mstruebing/editorconfig-checker:$image_version \
-      sh -c "ec --exclude '.git/' $dry_run_opt \$($filter) /dev/null"
+      sh -c "ec --exclude '.git/' ${dry_run_opt:-} \$($filter) /dev/null"
 }
 
+# ==============================================================================
+
 function is-arg-true() {
 
   if [[ "$1" =~ ^(true|yes|y|on|1|TRUE|YES|Y|ON)$ ]]; then
@@ -89,9 +92,8 @@ function is-arg-true() {
 
 # ==============================================================================
 
+is-arg-true "${VERBOSE:-false}" && set -x
 
-is-arg-true "$VERBOSE" && set -x
-
-main $*
+main "$@"
 
 exit 0