Merge branch 'main' into feat/spark-no-jmx-exporter

razvan · web-flow · commit 198db55957d3 · 2025-06-12T14:58:21.000+02:00
diff --git a/.github/ISSUE_TEMPLATE/early-pre-release.md b/.github/ISSUE_TEMPLATE/early-pre-release.md
@@ -59,5 +59,5 @@ Part of stackabletech/issues#xxx.
 - [ ] testing-tools
 - [ ] statsd_exporter
 - [ ] cyclonedx-bom (pip)
-- [ ] cargo-cyclonedx (Look for: `CARGO_CYCLONEDX_CRATE_VERSION`)
-- [ ] cargo-auditable (Look for: `CARGO_AUDITABLE_CRATE_VERSION`)
+- [ ] csi-provisioner for secret-operator
+- [ ] csi-provisioner for listener-operator
diff --git a/.github/ISSUE_TEMPLATE/update-base-stackable.md b/.github/ISSUE_TEMPLATE/update-base-stackable.md
@@ -27,12 +27,17 @@ Add/Change/Remove anything that isn't applicable anymore
 
 ## Update tasks
 
-- [ ] Update UBI version hash in the Dockerfile (`FROM`)
+### `stackable-base`
+
+- [ ] Update `FROM ...ubi-minimal` version hash in the Dockerfile
+- [ ] Update `CONFIG_UTILS_VERSION`
+
+### `stackable-devel`
+
+- [ ] Update `FROM ...ubi-minimal` version hash in the Dockerfile
 - [ ] Update `RUST_DEFAULT_TOOLCHAIN_VERSION`
 - [ ] Update `CARGO_CYCLONEDX_CRATE_VERSION`
 - [ ] Update `CARGO_AUDITABLE_CRATE_VERSION`
-- [ ] Update `PROTOC_VERSION`
-- [ ] Update `CONFIG_UTILS_VERSION`
 
 ## Related Pull Requests
 
diff --git a/.github/ISSUE_TEMPLATE/update-product-druid.md b/.github/ISSUE_TEMPLATE/update-product-druid.md
@@ -27,12 +27,13 @@ Add/Change/Remove anything that isn't applicable anymore
 
 ## Update tasks
 
+- [ ] Release a new version of [druid-opa-authorizer](https://github.com/stackabletech/druid-opa-authorizer)
 - [ ] Update `versions.py` to reflect the agreed upon versions in the spreadsheet (including the removal of old versions).
+- [ ] Update the [druid-opa-authorizer](https://github.com/stackabletech/druid-opa-authorizer/) with the new set of versions.
 - [ ] Upload new version (see `druid/upload_new_druid_version.sh`).
 - [ ] Create a file: `druid/stackable/patches/x.y.z/.gitkeep`, add patches if applicable.
 - [ ] Delete old patch directories.
 - [ ] Update `versions.py` to the latest supported version of JVM (base and devel).
-- [ ] Update the [druid-opa-authorizer](https://github.com/stackabletech/druid-opa-authorizer/) with the new set of versions.
 - [ ] Check other operators (getting_started / kuttl / supported-versions) for usage of the versions. Add the PR(s) to the list below.
 - [ ] Ensure prerequisites are up to date (required-external-components.adoc).
 - [ ] Update the version in demos. Add the PR(s) to the list below.
@@ -43,11 +44,11 @@ Add/Change/Remove anything that isn't applicable anymore
 > Delete any items that do not apply so that all applicable items can be checked.
 > For example, if you add release notes to the documentation repository, you do not need the latter two criteria.
 
+- [ ] _Link to [druid-opa-authorizer](https://github.com/stackabletech/druid-opa-authorizer/) PR_
 - [ ] _Link to the docker-images PR (product update)_
 - [ ] _Link to the operator PR (getting_started / kuttl / supported-versions)_
 - [ ] _Link to any other operator PRs (getting_started / kuttl)_
 - [ ] _Link to demo PR (raise against the `main` branch)_
-- [ ] _Link to [druid-opa-authorizer](https://github.com/stackabletech/druid-opa-authorizer/) PR_
 - [ ] _Link to the Release Notes PR in the documentation repo (if not a comment below)_
 
 ## Acceptance
diff --git a/.github/ISSUE_TEMPLATE/update-product-trino.md b/.github/ISSUE_TEMPLATE/update-product-trino.md
@@ -33,6 +33,7 @@ Add/Change/Remove anything that isn't applicable anymore
 - [ ] Update `versions.py` to the latest supported version of JVM (base and devel).
 - [ ] Update other dependencies if applicable (eg: jmx_exporter, opa_authorizer, storage_connector, etc).
 - [ ] Check other operators (getting_started / kuttl / supported-versions) for usage of the versions. Add the PR(s) to the list below.
+- [ ] Update versions in `tests/templates/kuttl/opa-authorization/check-opa.py.j2`
 - [ ] Update the version in demos. Add the PR(s) to the list below.
 
 ### trino-cli
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -63,7 +63,7 @@ All notable changes to this project will be documented in this file.
 - stackable-devel: Bump ubi9 base image ([#1103], [#1137]).
 - spark-k8s: Include spark-connect jars, replace OpenJDK with Temurin JDK, cleanup ([#1034]).
 - spark-connect-client: Image is now completely based on spark-k8s and includes JupyterLab and other demo dependencies ([#1071]).
-- jmx_exporter: Bump products to use `1.2.0` ([#1090]).
+- jmx_exporter: Bump products to use `1.3.0` ([#1090], [#1156]).
 - kubectl: Bump products to use `1.33.0` ([#1090]).
 - yq: Bump products to use `4.45.2` ([#1090]).
 - cyclonedx-bom: Bump airflow and superset to use `6.0.0` ([#1090]).
@@ -181,6 +181,7 @@ All notable changes to this project will be documented in this file.
 [#1150]: https://github.com/stackabletech/docker-images/pull/1150
 [#1151]: https://github.com/stackabletech/docker-images/pull/1151
 [#1152]: https://github.com/stackabletech/docker-images/pull/1152
+[#1156]: https://github.com/stackabletech/docker-images/pull/1156
 
 ## [25.3.0] - 2025-03-21
 
diff --git a/airflow/stackable/constraints/3.0.1/constraints-python3.12.txt b/airflow/stackable/constraints/3.0.1/constraints-python3.12.txt
@@ -122,7 +122,14 @@ apache-airflow-providers-asana==2.9.1
 apache-airflow-providers-atlassian-jira==3.0.2
 apache-airflow-providers-celery==3.10.6
 apache-airflow-providers-cloudant==4.1.1
-apache-airflow-providers-cncf-kubernetes==10.4.3
+# Stackable patch:
+# The 10.4.3 Kubernetes provider shipped with Airflow 3.0.1 has a problem.
+# https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/10.5.0/changelog.html#id1
+# It uses a busybox sidecar container to start tasks using the kubernetes executor
+# Two problems with that:
+# 1) The securityContext says "runAsNonRoot" but the busybox image defaults to root so it fails to be scheduled (at least on a standard OpenShift)
+# 2) The busybox image might not be available on air-gapped clusters
+apache-airflow-providers-cncf-kubernetes==10.5.0
 apache-airflow-providers-cohere==1.4.3
 apache-airflow-providers-common-compat==1.6.1
 apache-airflow-providers-common-io==1.5.4
diff --git a/hadoop/versions.py b/hadoop/versions.py
@@ -5,7 +5,7 @@
         "java-base": "11",
         "java-devel": "11",
         "async_profiler": "2.9",
-        "jmx_exporter": "1.2.0",
+        "jmx_exporter": "1.3.0",
         "protobuf": "3.7.1",
         "hdfs_utils": "0.4.0",
     },
@@ -14,7 +14,7 @@
         "java-base": "11",
         "java-devel": "11",
         "async_profiler": "2.9",
-        "jmx_exporter": "1.2.0",
+        "jmx_exporter": "1.3.0",
         "protobuf": "3.7.1",
         "hdfs_utils": "0.4.1",
     },
diff --git a/hive/versions.py b/hive/versions.py
@@ -1,7 +1,7 @@
 versions = [
     {
         "product": "3.1.3",
-        "jmx_exporter": "1.2.0",
+        "jmx_exporter": "1.3.0",
         # Hive 3 must be built with Java 8 but will run on Java 11
         "java-base": "11",
         "java-devel": "8",
@@ -13,7 +13,7 @@
     },
     {
         "product": "4.0.0",
-        "jmx_exporter": "1.2.0",
+        "jmx_exporter": "1.3.0",
         # Hive 4 must be built with Java 8 (according to GitHub README) but seems to run on Java 11
         "java-base": "11",
         "java-devel": "8",
@@ -25,7 +25,7 @@
     },
     {
         "product": "4.0.1",
-        "jmx_exporter": "1.2.0",
+        "jmx_exporter": "1.3.0",
         # Hive 4 must be built with Java 8 (according to GitHub README) but seems to run on Java 11
         "java-base": "11",
         "java-devel": "8",
diff --git a/kafka/versions.py b/kafka/versions.py
@@ -6,7 +6,7 @@
         "scala": "2.13",
         "kafka/kcat": "1.7.0",
         "opa_authorizer": "1.5.1",
-        "jmx_exporter": "1.2.0",
+        "jmx_exporter": "1.3.0",
     },
     {
         "product": "3.9.0",
@@ -15,7 +15,7 @@
         "scala": "2.13",
         "kafka/kcat": "1.7.0",
         "opa_authorizer": "1.5.1",
-        "jmx_exporter": "1.2.0",
+        "jmx_exporter": "1.3.0",
     },
     {
         "product": "3.9.1",
@@ -24,7 +24,7 @@
         "scala": "2.13",
         "kafka/kcat": "1.7.0",
         "opa_authorizer": "1.5.1",
-        "jmx_exporter": "1.2.0",
+        "jmx_exporter": "1.3.0",
     },
     {
         "product": "4.0.0",
@@ -33,6 +33,6 @@
         "scala": "2.13",
         "kafka/kcat": "1.7.0",
         "opa_authorizer": "1.5.1",
-        "jmx_exporter": "1.2.0",
+        "jmx_exporter": "1.3.0",
     },
 ]
diff --git a/omid/versions.py b/omid/versions.py
@@ -3,12 +3,12 @@
         "product": "1.1.2",
         "java-base": "11",
         "java-devel": "11",
-        "jmx_exporter": "1.2.0",
+        "jmx_exporter": "1.3.0",
     },
     {
         "product": "1.1.3",
         "java-base": "11",
         "java-devel": "11",
-        "jmx_exporter": "1.2.0",
+        "jmx_exporter": "1.3.0",
     },
 ]
diff --git a/stackable-base/Dockerfile b/stackable-base/Dockerfile
@@ -1,6 +1,12 @@
 # syntax=docker/dockerfile:1.16.0@sha256:e2dd261f92e4b763d789984f6eab84be66ab4f5f08052316d8eb8f173593acf7
 # check=error=true
 
+#
+# Base image for final stages
+# Should not contain any components that are not needed (or helpful) at runtime
+# Build time dependencies that are used in multiple images can be added to `stackable-devel` (or a derivative like `java-devel`) instead
+#
+
 FROM stackable/image/stackable-devel AS config-utils
 
 # Find the latest version here: https://github.com/stackabletech/config-utils/tags
diff --git a/stackable-devel/Dockerfile b/stackable-devel/Dockerfile
@@ -3,6 +3,9 @@
 
 #
 # Base image for builder stages
+# It is intended to be used as an intermediate "workbench" layer to build components
+# It should not be the base for a final image to avoid shipping build time dependencies like `patchable` or `gcc`
+# Use `stackable-base` as a base for the final image stage instead
 #
 
 # Find the latest version at https://catalog.redhat.com/software/containers/ubi9/ubi-minimal/615bd9b4075b022acc111bf5?container-tabs=gti
diff --git a/trino/Dockerfile b/trino/Dockerfile
@@ -27,6 +27,20 @@ cd "$(/stackable/patchable --images-repo-root=src checkout trino ${PRODUCT})"
 # Create snapshot of the source code including custom patches
 tar -czf /stackable/trino-${PRODUCT}-src.tar.gz .
 
+# Trino is using something (git-commit-id-plugin in the past, maybe something else now) that is
+# reading the Git history and searches for a tag to pull the version from. It sounds weird to me
+# why someone would do that over just picking the version from the pom.xml, but they propably
+# have their reasons. See e.g. https://github.com/trinodb/trino/discussions/18963.
+# So we fake it till we make it and create a Git repo and the correct tag. The trino-operator
+# smoke test checks that "select version()" is working.
+# Also, we need to initialize a new Git repo because `git-commit-id-maven-plugin` has a problem with worktrees, see https://github.com/git-commit-id/git-commit-id-maven-plugin/issues/215
+rm .git
+git init
+git config user.email "fake.commiter@stackable.tech"
+git config user.name "Fake commiter"
+git commit --allow-empty --message "Fake commit, so that we can create a tag"
+git tag ${PRODUCT}
+
 SKIP_PROJECTS="!docs"
 if [ "$PRODUCT" = "470" ] || [ "$PRODUCT" = "451" ]; then
   SKIP_PROJECTS="$SKIP_PROJECTS,!core/trino-server-rpm"
@@ -41,7 +55,6 @@ fi
   -DskipTests `# Skip test execution` \
   -Dcheckstyle.skip `# Skip checkstyle checks. We dont care if the code is properly formatted, it just wastes time` \
   -Dmaven.javadoc.skip=true `# Dont generate javadoc` \
-  -Dmaven.gitcommitid.skip=true `# The gitcommitid plugin cannot work with git workspaces (ie: patchable)` \
   --projects="$SKIP_PROJECTS"
 
 # Delete intermediate build products to free some space and keep runners happy
diff --git a/trino/versions.py b/trino/versions.py
@@ -3,21 +3,21 @@
         "product": "451",
         "java-base": "22",
         "java-devel": "22",
-        "jmx_exporter": "1.2.0",
+        "jmx_exporter": "1.3.0",
         "trino/storage-connector": "451",
     },
     {
         "product": "470",
         "java-base": "23",
         "java-devel": "23",
-        "jmx_exporter": "1.2.0",
+        "jmx_exporter": "1.3.0",
         "trino/storage-connector": "470",
     },
     {
         "product": "476",
         "java-base": "24",
         "java-devel": "24",
-        "jmx_exporter": "1.2.0",
+        "jmx_exporter": "1.3.0",
         "trino/storage-connector": "476",
     },
 ]
diff --git a/zookeeper/versions.py b/zookeeper/versions.py
@@ -7,6 +7,6 @@
         # zookeeper: Execution spotbugs of goal com.github.spotbugs:spotbugs-maven-plugin:4.0.0:spotbugs failed: Java
         # returned: 1 -> [Help 1]
         "java-devel": "11",
-        "jmx_exporter": "1.2.0",
+        "jmx_exporter": "1.3.0",
     },
 ]
