Merge branch 'main' into feat/spark-4

Author: razvan

.cargo/config.toml

@@ -0,0 +1,2 @@
+[alias]
+patchable = ["run", "--bin", "patchable", "--"]

.gitattributes

@@ -0,0 +1 @@
+nix/** linguist-generated

.github/ISSUE_TEMPLATE/add-product.md

@@ -0,0 +1,29 @@
+---
+name: Add new Product
+about: >-
+  This template contains instructions specific to adding a new product.
+title: >-
+  tracking: Add new product <PRODUCT>
+labels: []
+# Currently, projects cannot be assigned via front-matter.
+projects: ['stackabletech/10']
+assignees: ''
+---
+
+## Tasks
+
+- [ ] Create a new top-level folder for the product. The name of the folder must
+      use the lowercase product name.
+- [ ] Create a README.md file outlining special considerations required to
+      update or run the product. See existing README files as a guide of
+      reference.
+- [ ] Add a `versions.py` file to the folder. Add all required key-value pairs.
+- [ ] Add a new "Update Product" issue template in `.github/ISSUE_TEMPLATE/`
+      folder. See existing ones as a guide of reference.
+- [ ] Add a new `build_<PRODUCT>.yml` GitHub Action workflow in the
+      `.github/workflows` folder. Use existing local action whenever possible
+      or consider creating a new one when there is no fitting action available.
+- [ ] Run `.scripts/update_readme_badges.sh` to generate the new status badge.
+
+_Please consider updating this template if these instructions are wrong, or
+could be made clearer._

.github/ISSUE_TEMPLATE/early-pre-release.md

@@ -0,0 +1,61 @@
+---
+name: Early Pre-Release Container Image Updates
+about: This template can be used to track the container image updates leading up to the next Stackable release
+title: "chore: Update Container Images for Stackable Release YY.M.X"
+labels: ['epic']
+assignees: ''
+---
+
+<!--
+    DO NOT REMOVE THIS COMMENT. It is intended for people who might copy/paste from the previous release issue.
+    This was created by an issue template: https://github.com/stackabletech/docker-images/issues/new/choose.
+-->
+
+<!-- Update this with the parent tracking issue for the release -->
+Part of stackabletech/issues#xxx.
+
+> [!NOTE]
+> Update the product versions based on what has been decided upon in the _Product Spreadsheet[^1]_.
+
+[^1]: Currently this is a private spreadsheet
+
+> [!IMPORTANT]
+> Replace the items in the task lists below with the subsequent tracking issue.
+
+## Product Container Images
+
+<!--
+    Find templates for bases/products:
+
+    find .github/ISSUE_TEMPLATE/update-*.md -printf "%f\n" \
+    | sort \
+    | xargs -I {} echo "- [ ] [Create issue from template: {}](https://github.com/stackabletech/docker-images/issues/new?template={})"
+-->
+
+<!-- todo: consider removing the ubi*-rust-builder from the release process. -->
+
+- [ ] [Create issue from template: update-base-java.md](https://github.com/stackabletech/docker-images/issues/new?template=update-base-java.md)
+- [ ] [Create issue from template: update-base-stackable.md](https://github.com/stackabletech/docker-images/issues/new?template=update-base-stackable.md)
+- [ ] [Create issue from template: update-base-vector.md](https://github.com/stackabletech/docker-images/issues/new?template=update-base-vector.md)
+- [ ] [Create issue from template: update-product-airflow.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-airflow.md)
+- [ ] [Create issue from template: update-product-druid.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-druid.md)
+- [ ] [Create issue from template: update-product-hbase-phoenix-omid.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-hbase-phoenix-omid.md)
+- [ ] [Create issue from template: update-product-hdfs.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-hdfs.md)
+- [ ] [Create issue from template: update-product-hive.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-hive.md)
+- [ ] [Create issue from template: update-product-kafka.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-kafka.md)
+- [ ] [Create issue from template: update-product-nifi.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-nifi.md)
+- [ ] [Create issue from template: update-product-opa.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-opa.md)
+- [ ] [Create issue from template: update-product-spark.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-spark.md)
+- [ ] [Create issue from template: update-product-superset.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-superset.md)
+- [ ] [Create issue from template: update-product-trino.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-trino.md)
+- [ ] [Create issue from template: update-product-zookeeper.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-zookeeper.md)
+
+## Additional items which don't have a tracking issue
+
+- [ ] jmx_exporter (validate via hdfs-operator smoke tests)
+- [ ] krb5
+- [ ] tools (update the versions in version.py)
+- [ ] testing-tools (update the base, maybe keycloak)
+- [ ] statsd_exporter
+- [ ] csi-provisioner for secret-operator
+- [ ] csi-provisioner for listener-operator

.github/ISSUE_TEMPLATE/pre-release.md

@@ -0,0 +1,83 @@
+---
+name: Update Java Bases
+about: >-
+  This template contains instructions specific to updating this product and/or
+  container image(s).
+title: >-
+  chore(java-bases): Update container images ahead of Stackable Release YY.M.X
+labels: []
+# Currently, projects cannot be assigned via front-matter.
+projects: ['stackabletech/10']
+assignees: ''
+---
+
+Part of #xxx.
+
+<!--
+This gives hints to the person doing the work.
+Add/Change/Remove anything that isn't applicable anymore
+-->
+- Add: `x.x.x`
+- Remove: `y.y.y`
+
+> [!TIP]
+> Please add the `scheduled-for/20XX-XX` label, and add to the [Stackable Engineering][1] project.
+>
+> [1]: https://github.com/orgs/stackabletech/projects/10
+
+<!-- markdownlint-disable-next-line MD028 -->
+> [!IMPORTANT]
+>
+> - `java-base` uses [vector](https://github.com/stackabletech/docker-images/blob/main/vector/Dockerfile) and is used as a base for java products.
+> - `java-devel` uses [stackable-base](https://github.com/stackabletech/docker-images/blob/main/stackable-base/Dockerfile) and is used to build java products.
+
+Typically product updates will determine which version of Java is required, but
+we should also make new versions of Java available for use.
+
+> [!TIP]
+> You can search for available java versions at [rpmfind.net], and search the
+> term `openjdk-headless`.
+> _It isn't perfect, as it will depend on what is available via microdnf._
+
+## Update tasks
+
+- [ ] Add any new versions of java to both `java-base/versions.py` and `java-devel/versions.py`
+- [ ] Remove versions when there are no long any references (eg: `grep java- **/versions.py | grep "1.8.0"`)
+
+## Related Pull Requests
+
+- _Link to the docker-images PR (product update)_
+
+## Acceptance
+
+> [!TIP]
+> This list should be completed by the assignee(s), once respective PRs have been merged. Once all items have been
+> checked, the issue can be moved into _Development: Done_.
+
+- [ ] Can build a product image that uses the new version(s)
+- [ ] Both `java-base` and `java-devel` have the same Java versions in `versions.py`
+- [ ] Kuttl smoke test passes locally for a product using the new Java version
+
+<details>
+<summary>Testing instructions</summary>
+
+```shell
+# See the latest version at https://pypi.org/project/image-tools-stackabletech/
+pip install image-tools-stackabletech==0.0.16
+
+# Test a product image can build, eg: ZooKeeper
+bake --product zookeeper=x.y.z # where x.y.z is a valid product version using the newly added Java version
+
+kind load docker-image oci.stackable.tech/sdp/zookeeper:x.y.z-stackable0.0.0-dev
+
+# Change directory into one of the operator repositories (eg: zookeeper-operator) and update the
+# product version in tests/test-definition.yaml
+./scripts/run-tests --test-suite smoke-latest # or similar
+```
+
+</details>
+
+_Please consider updating this template if these instructions are wrong, or
+could be made clearer._
+
+[rpmfind.net]: https://rpmfind.net/linux/RPM/Development_Java.html

.github/ISSUE_TEMPLATE/update-base-java.md

@@ -0,0 +1,67 @@
+---
+name: Update Stackable Base
+about: >-
+  This template contains instructions specific to updating this product and/or
+  container image(s).
+title: >-
+  chore(stackable-base): Update container images ahead of Stackable Release YY.M.X
+labels: []
+# Currently, projects cannot be assigned via front-matter.
+projects: ['stackabletech/10']
+assignees: ''
+---
+
+Part of #xxx.
+
+> [!TIP]
+> Please add the `scheduled-for/20XX-XX` label, and add to the [Stackable Engineering][1] project.
+>
+> [1]: https://github.com/orgs/stackabletech/projects/10
+
+## Update tasks
+
+> [!NOTE]
+> When updating the base image, you will likely get a build failure related to the CA certificates.
+> This means you will need to update the `ca-certificates-*` package and try again. The build will
+> fail if the blocked certificates are still found.
+> The package check exists so that we can remove it once (if ever) the _bad_ CA has been removed.
+
+### `stackable-base`
+
+- [ ] Update `FROM ...ubi-minimal` version hash in the Dockerfile
+- [ ] Update `CONFIG_UTILS_VERSION`
+
+### `stackable-devel`
+
+- [ ] Update `FROM ...ubi-minimal` version hash in the Dockerfile
+- [ ] Update `RUST_DEFAULT_TOOLCHAIN_VERSION` (if tools need it, eg: patchable, config-utils)
+- [ ] Update `CARGO_CYCLONEDX_CRATE_VERSION` (if necessary)
+- [ ] Update `CARGO_AUDITABLE_CRATE_VERSION` (if necessary)
+
+## Related Pull Requests
+
+- _Link to the docker-images PR (product update)_
+
+## Acceptance
+
+> [!TIP]
+> This list should be completed by the assignee(s), once respective PRs have been merged. Once all items have been
+> checked, the issue can be moved into _Development: Done_.
+
+- [ ] Can build the image locally
+- [ ] Can build the vector image
+
+<details>
+<summary>Testing instructions</summary>
+
+```shell
+# See the latest version at https://pypi.org/project/image-tools-stackabletech/
+pip install image-tools-stackabletech==0.0.16
+
+bake --product vector=x.y.z # where x.y.z is a valid version
+```
+
+</details>
+
+_Please consider updating this template if these instructions are wrong, or
+could be made clearer._

.github/ISSUE_TEMPLATE/update-base-stackable.md

@@ -0,0 +1,71 @@
+---
+name: Update UBI Rust Builders
+about: >-
+  This template contains instructions specific to updating this product and/or
+  container image(s).
+title: >-
+  chore(ubi-rust-builders): Update container images ahead of Stackable Release YY.M.X
+labels: []
+# Currently, projects cannot be assigned via front-matter.
+projects: ['stackabletech/10']
+assignees: ''
+---
+
+<!--
+I don't think we should tie rust bumps to releases - they should be done as
+developers need newer versions , which could be multiple times in a release.
+If there are no bumps in a release, we can still rely on SecObserve and Renovate
+to alert us to security vulnerabilities.
+-->
+Part of #xxx.
+
+<!--
+This gives hints to the person doing the work.
+Add/Change/Remove anything that isn't applicable anymore
+-->
+- Add: `x.x.x`
+- Remove: `y.y.y`
+
+> [!TIP]
+> Please add the `scheduled-for/20XX-XX` label, and add to the [Stackable Engineering][1] project.
+>
+> [1]: https://github.com/orgs/stackabletech/projects/10
+
+## Update tasks
+
+- [ ] Update UBI version hash in the Dockerfile (`FROM`)
+- [ ] Update `RUST_DEFAULT_TOOLCHAIN_VERSION`
+- [ ] Update `CARGO_CYCLONEDX_CRATE_VERSION`
+- [ ] Update `CARGO_AUDITABLE_CRATE_VERSION`
+- [ ] Update `PROTOC_VERSION`
+
+## Related Pull Requests
+
+- _Link to the docker-images PR (product update)_
+- _Bump rust toolchain in operator-rs_
+- _Bump rust toolchain in operator-templating_
+
+## Acceptance
+
+> [!TIP]
+> This list should be completed by the assignee(s), once respective PRs have been merged. Once all items have been
+> checked, the issue can be moved into _Development: Done_.
+
+- [ ] Done for [ubi9-rust-builder/Dockerfile](https://github.com/stackabletech/docker-images/blob/main/ubi9-rust-builder/Dockerfile)
+- [ ] Can build the image locally
+- [ ] Can build an operator image
+
+<details>
+<summary>Testing instructions</summary>
+
+```shell
+docker build -t oci.stackable.tech/sdp/ubi9-rust-builder . -f ubi9-rust-builder/Dockerfile
+
+# Change directory into the an operator repository and ensure the image can build
+docker build . -f docker/Dockerfile
+```
+
+</details>
+
+_Please consider updating this template if these instructions are wrong, or
+could be made clearer._