Merge branch 'spike/bundle-builder-v2' into spike/userinfofetcher-regorules

Author: nightkr

.github/ISSUE_TEMPLATE/new_version.md

@@ -7,25 +7,25 @@ assignees: ''
 
 ---
 
-**Which new version of OpenPolicyAgent should we support?**
+## Which new version of OpenPolicyAgent should we support?
 
 Please specify the version, version range or version numbers to support, please also add these to the issue title
 
-**Additional information**
+## Additional information
 
 If possible, provide a link to release notes/changelog
 
-**Changes required**
+## Changes required
 
 Are there any upstream changes that we need to support?
 e.g. new features, changed features, deprecated features etc.
 
+## Implementation checklist
 
-
-**Implementation checklist**
-
-Please don't change anything in this list.
-Not all of these steps are necessary for all versions.
+<!--
+    Please don't change anything in this list.
+    Not all of these steps are necessary for all versions.
+-->
 
 - [ ] Update the Docker image
 - [ ] Update documentation to include supported version(s)

.github/PULL_REQUEST_TEMPLATE/pre-release-getting-started-script.md

@@ -0,0 +1,34 @@
+# Check and Update Getting Started Script
+
+<!--
+    Make sure to update the link in 'issues/.github/ISSUE_TEMPLATE/pre-release-getting-started-scripts.md'
+    when you rename this file.
+-->
+
+<!--
+    Replace 'TRACKING_ISSUE' with the applicable release tracking issue number.
+-->
+
+Part of <https://github.com/stackabletech/issues/issues/TRACKING_ISSUE>
+
+> [!NOTE]
+> During a Stackable release we need to check (and optionally update) the
+> getting-started scripts to ensure they still work after product and operator
+> updates.
+
+```shell
+# Some of the scripts are in a code/ subdirectory
+# pushd docs/modules/superset/examples/getting_started
+# pushd docs/modules/superset/examples/getting_started/code
+pushd $(fd -td getting_started | grep examples); cd code 2>/dev/null || true
+
+# Make a fresh cluster (~12 seconds)
+kind delete cluster && kind create cluster
+./getting_started.sh stackablectl
+
+# Make a fresh cluster (~12 seconds)
+kind delete cluster && kind create cluster
+./getting_started.sh helm
+
+popd
+```

.github/PULL_REQUEST_TEMPLATE/pre-release-rust-deps.md

@@ -0,0 +1,38 @@
+# Bump Rust Dependencies for Stackable Release XX.(X)X
+
+<!--
+    Make sure to update the link in 'issues/.github/ISSUE_TEMPLATE/pre-release-operator-rust-deps.md'
+    when you rename this file.
+-->
+
+<!--
+    Replace 'TRACKING_ISSUE' with the applicable release tracking issue number.
+-->
+
+Part of <https://github.com/stackabletech/issues/issues/TRACKING_ISSUE>
+
+> [!NOTE]
+> During a Stackable release we need to update various Rust dependencies before
+> entering the final release period to ensure we run the latest versions of
+> crates. These bumps also include previously updated and released crates from
+> the `operator-rs` repository.
+
+```[tasklist]
+### Tasks
+- [ ] Bump Rust Dependencies, see below for more details.
+- [ ] Add changelog entry stating which important crates were bumped (including the version).
+```
+
+> [!NOTE]
+> The bumping / updating of Rust dependencies is done in multiple steps:
+>
+> 1. Update the minimum Version in the root `Cargo.toml` manifest.
+> 2. Run the `cargo update` command, which also updates the `Cargo.lock` file.
+> 3. Lastly, run `make regenerate-nix` to update the `Cargo.nix` file.
+
+```[tasklist]
+### Bump Rust Dependencies
+- [ ] Bump `stackable-operator` and friends.
+- [ ] Bump `product-version`.
+- [ ] Bump all other dependencies.
+```

.github/pull_request_template.md

@@ -11,7 +11,7 @@
 # Author
 - [ ] Changes are OpenShift compatible
 - [ ] CRD changes approved
-- [ ] CRD documentation for all fields, following the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs-style-guide).
+- [ ] CRD documentation for all fields, following the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs/style-guide).
 - [ ] Helm chart can be installed and deployed operator works
 - [ ] Integration tests passed (for non trivial changes)
 - [ ] Changes need to be "offline" compatible
@@ -22,7 +22,7 @@
 - [ ] Code contains useful comments
 - [ ] Code contains useful logging statements
 - [ ] (Integration-)Test cases added
-- [ ] Documentation added or updated. Follows the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs-style-guide).
+- [ ] Documentation added or updated. Follows the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs/style-guide).
 - [ ] Changelog updated
 - [ ] Cargo.toml only contains references to git tags (not specific commits or branches)
 ```

.github/workflows/build.yml

@@ -25,7 +25,7 @@ env:
   CARGO_TERM_COLOR: always
   CARGO_INCREMENTAL: '0'
   CARGO_PROFILE_DEV_DEBUG: '0'
-  RUST_TOOLCHAIN_VERSION: "1.77.2"
+  RUST_TOOLCHAIN_VERSION: "1.80.0"
   RUSTFLAGS: "-D warnings"
   RUSTDOCFLAGS: "-D warnings"
   RUST_LOG: "info"
@@ -44,8 +44,8 @@ jobs:
       - name: Install host dependencies
         uses: awalsh128/cache-apt-pkgs-action@a6c3917cc929dd0345bfb2d3feaf9101823370ad # v1.4.2
         with:
-          packages: protobuf-compiler krb5-user libkrb5-dev libclang-dev liblzma-dev libssl-dev pkg-config
-          version: 1.0
+          packages: protobuf-compiler krb5-user libkrb5-dev libclang-dev liblzma-dev libssl-dev pkg-config apt-transport-https
+          version: ubuntu-latest
       - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
         with:
           submodules: recursive
@@ -57,7 +57,7 @@ jobs:
           key: udeps
           cache-all-crates: "true"
       - uses: stackabletech/cargo-install-action@cargo-udeps
-      - run: cargo udeps --workspace
+      - run: cargo udeps --workspace --all-targets
 
   # This job evaluates the github environment to determine why this action is running and selects the appropriate
   # target repository for published Helm charts based on this.
@@ -142,8 +142,8 @@ jobs:
       - name: Install host dependencies
         uses: awalsh128/cache-apt-pkgs-action@a6c3917cc929dd0345bfb2d3feaf9101823370ad # v1.4.2
         with:
-          packages: protobuf-compiler krb5-user libkrb5-dev libclang-dev liblzma-dev libssl-dev pkg-config
-          version: 1.0
+          packages: protobuf-compiler krb5-user libkrb5-dev libclang-dev liblzma-dev libssl-dev pkg-config apt-transport-https
+          version: ubuntu-latest
       - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
         with:
           submodules: recursive
@@ -177,8 +177,8 @@ jobs:
       - name: Install host dependencies
         uses: awalsh128/cache-apt-pkgs-action@a6c3917cc929dd0345bfb2d3feaf9101823370ad # v1.4.2
         with:
-          packages: protobuf-compiler krb5-user libkrb5-dev libclang-dev liblzma-dev libssl-dev pkg-config
-          version: 1.0
+          packages: protobuf-compiler krb5-user libkrb5-dev libclang-dev liblzma-dev libssl-dev pkg-config apt-transport-https
+          version: ubuntu-latest
       - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
         with:
           submodules: recursive
@@ -199,8 +199,8 @@ jobs:
       - name: Install host dependencies
         uses: awalsh128/cache-apt-pkgs-action@a6c3917cc929dd0345bfb2d3feaf9101823370ad # v1.4.2
         with:
-          packages: protobuf-compiler krb5-user libkrb5-dev libclang-dev liblzma-dev libssl-dev pkg-config
-          version: 1.0
+          packages: protobuf-compiler krb5-user libkrb5-dev libclang-dev liblzma-dev libssl-dev pkg-config apt-transport-https
+          version: ubuntu-latest
       - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
         with:
           submodules: recursive
@@ -257,8 +257,8 @@ jobs:
       - name: Install host dependencies
         uses: awalsh128/cache-apt-pkgs-action@a6c3917cc929dd0345bfb2d3feaf9101823370ad # v1.4.2
         with:
-          packages: protobuf-compiler krb5-user libkrb5-dev libclang-dev liblzma-dev libssl-dev pkg-config
-          version: 1.0
+          packages: protobuf-compiler krb5-user libkrb5-dev libclang-dev liblzma-dev libssl-dev pkg-config apt-transport-https
+          version: ubuntu-latest
       - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
         with:
           submodules: recursive
@@ -326,8 +326,8 @@ jobs:
       - name: Install host dependencies
         uses: awalsh128/cache-apt-pkgs-action@a6c3917cc929dd0345bfb2d3feaf9101823370ad # v1.4.2
         with:
-          packages: protobuf-compiler krb5-user libkrb5-dev libclang-dev liblzma-dev libssl-dev pkg-config
-          version: 1.0
+          packages: protobuf-compiler krb5-user libkrb5-dev libclang-dev liblzma-dev libssl-dev pkg-config apt-transport-https
+          version: ${{ matrix.runner }}
       - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
         with:
           submodules: recursive
@@ -359,7 +359,6 @@ jobs:
           # Installing helm and yq on ubicloud-standard-8-arm only
           if [ "$(arch)" = "aarch64" ]; then
             curl https://baltocdn.com/helm/signing.asc | gpg --dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null
-            sudo apt-get -y install apt-transport-https
             echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list
             sudo apt-get -y update
             sudo apt-get -y install helm
@@ -369,7 +368,13 @@ jobs:
           make -e build
       - name: Publish Docker image and Helm chart
         if: ${{ !github.event.pull_request.head.repo.fork }}
-        run: make -e publish
+        run: |
+          # We want to publish helmcharts only once as they have a common name, while still publishing both images with architecture specific tags
+          if [ "$(uname -m)" = "x86_64" ]; then
+            make -e publish
+          else
+            make -e docker-publish
+          fi
         # Output the name of the published image to the Job output for later use
       - id: printtag
         name: Output image name and tag

.github/workflows/pr_pre-commit.yaml

@@ -0,0 +1,17 @@
+---
+name: pre-commit
+
+on:
+  pull_request:
+
+jobs:
+  pre-commit:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
+      - uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0
+        with:
+          python-version: '3.12'
+      - uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1
+        with:
+          extra_args: "" # Disable --all-files until we have time to fix druid/stackable/bin/run-druid

.github/workflows/pr_reviewdog.yaml

@@ -23,17 +23,6 @@ jobs:
         with:
           github_token: ${{ secrets.GITHUB_TOKEN }}
 
-  flake8:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
-      - uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # tag=v5.1.0
-        with:
-          python-version: "3.12"
-      - uses: reviewdog/action-flake8@99c2cfecdbc9111ec223b85b08af0e13a9a098dc # v3.10.0
-        with:
-          github_token: ${{ secrets.GITHUB_TOKEN }}
-
   hadolint:
     runs-on: ubuntu-latest
     steps:

.markdownlint.yaml

@@ -14,6 +14,10 @@ MD013:
   # Number of characters for code blocks
   code_block_line_length: 9999
 
+# MD033/no-inline-html
+MD033:
+  allowed_elements: [h1, img, p]
+
 # MD024/no-duplicate-heading/no-duplicate-header - Multiple headings with the same content
 MD024:
   # Only check sibling headings

.pre-commit-config.yaml

@@ -1,4 +1,6 @@
 ---
+exclude: ^(Cargo\.nix|crate-hashes\.json|nix/.*)$
+
 # See https://pre-commit.com for more information
 # See https://pre-commit.com/hooks.html for more hooks
 repos:
@@ -28,11 +30,25 @@ repos:
     rev: v0.40.0
     hooks:
       - id: markdownlint
+        types: [text]
+        files: \.md(\.j2)*$
+
+  - repo: https://github.com/koalaman/shellcheck-precommit
+    rev: v0.10.0
+    hooks:
+      - id: shellcheck
+        args: ["--severity=info"]
 
-  - repo: https://github.com/PyCQA/flake8
-    rev: 7.0.0
+  # WARNING (@NickLarsenNZ): Nix users need to install ruff first.
+  # If you do not, you will need to delete the cached ruff binary shown in the
+  # error message
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.5.1
     hooks:
-      - id: flake8
+      # Run the linter.
+      - id: ruff
+      # Run the formatter.
+      - id: ruff-format
 
   - repo: local
     hooks:

.readme/partials/borrowed/documentation.md.j2

@@ -1,3 +1,4 @@
+<!-- markdownlint-disable MD041 -->
 ## Documentation
 
 The stable documentation for this operator can be found [here](https://docs.stackable.tech/home/stable/{{operator_docs_slug}}).