feat: Add Opensearch image (#1215)

* feat(opensearch): Add OpenSearch image

* feat(opensearch): Change JDK to version 17

* feat(jdk-base): Create a jdk-base image for OpenSearch

* feat(opensearch): Upgrade to version 2.19.0

* feat(opensearch): Upgrade to version 2.19.1 and install the plugin repository-s3

* wip: build from source

* feat: SBOMs for OpenSearch and opensearch-security-plugin (#1211)

feat: SBOMs for opensearch

* fix file structure in image and make multi-arch compatible

* install performance analyzer plugin

* remove legacy apply-patches file

* add changelog entry

* build plugins from source and stop building snapshots

* replace performance analyzer with telemetry-otel & address PR feedback

* simplify build process

* address feedback in PR

* fix copypaste errors

* add stackable-devel to build dependencies

---------

Co-authored-by: Siegfried Weber <[email protected]>
Co-authored-by: Lukas Krug <[email protected]>

Author: 3 people

.github/ISSUE_TEMPLATE/early-pre-release.md

@@ -45,6 +45,7 @@ Part of stackabletech/issues#xxx.
 - [ ] [Create issue from template: update-product-kafka.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-kafka.md)
 - [ ] [Create issue from template: update-product-nifi.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-nifi.md)
 - [ ] [Create issue from template: update-product-opa.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-opa.md)
+- [ ] [Create issue from template: update-product-opensearch.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-opensearch.md)
 - [ ] [Create issue from template: update-product-spark.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-spark.md)
 - [ ] [Create issue from template: update-product-superset.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-superset.md)
 - [ ] [Create issue from template: update-product-trino.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-trino.md)

.github/ISSUE_TEMPLATE/update-product-opensearch.md

@@ -0,0 +1,79 @@
+---
+name: Update OpenSearch
+about: >-
+  This template contains instructions specific to updating this product and/or
+  container image(s).
+title: >-
+  chore(opensearch): Update container images ahead of Stackable Release YY.M.X
+labels: []
+# Currently, projects cannot be assigned via front-matter.
+projects: ['stackabletech/10']
+assignees: ''
+---
+
+Part of #xxx.
+
+<!--
+This gives hints to the person doing the work.
+Add/Change/Remove anything that isn't applicable anymore
+-->
+- Add: `x.x.x`
+- Remove: `y.y.y`
+
+> [!TIP]
+> Please add the `scheduled-for/20XX-XX` label, and add to the [Stackable Engineering][1] project.
+>
+> [1]: https://github.com/orgs/stackabletech/projects/10
+
+## Update tasks
+
+- [ ] Update `versions.py` to reflect the agreed upon versions in the spreadsheet (including the removal of old versions).
+- [ ] Update `versions.py` to the latest supported version of JVM (base and devel).
+- [ ] Update other dependencies if applicable (eg: security-plugin, etc).
+- [ ] Check other operators (getting_started / kuttl / supported-versions) for usage of the versions. Add the PR(s) to the list below.
+- [ ] Update the version in demos. Add the PR(s) to the list below.
+
+## Related Pull Requests
+
+> [!TIP]
+> Delete any items that do not apply so that all applicable items can be checked.
+> For example, if you add release notes to the documentation repository, you do not need the latter two criteria.
+
+- _Link to the docker-images PR (product update)_
+- _Link to the operator PR (getting_started / kuttl / supported-versions)_
+- _Link to any other operator PRs (getting_started / kuttl)_
+- _Link to demo PR (raise against the `main` branch)_
+- _Link to the Release Notes PR in the documentation repo (if not a comment below)_
+
+## Acceptance
+
+> [!TIP]
+> This list should be completed by the assignee(s), once respective PRs have been merged. Once all items have been
+> checked, the issue can be moved into _Development: Done_.
+
+- [ ] Can build image (either locally, or in CI)
+- [ ] Kuttl smoke tests passes (either locally, or in CI)
+- [ ] Release notes added to documentation and linked as a PR above
+- [ ] Release notes written in a comment below
+- [ ] Applicable `release-note` label added to this issue
+
+<details>
+<summary>Testing instructions</summary>
+
+```shell
+# See the latest version at https://pypi.org/project/image-tools-stackabletech/
+pip install image-tools-stackabletech==0.0.16
+
+bake --product opensearch=x.y.z # where x.y.z is the new version added in this PR
+
+kind load docker-image oci.stackable.tech/sdp/opensearch:x.y.z-stackable0.0.0-dev
+
+# Change directory into the opensearch-operator repository and update the
+# product version in tests/test-definition.yaml
+./scripts/run-tests --test-suite smoke-latest # or similar
+```
+
+</details>
+
+_Please consider updating this template if these instructions are wrong, or
+could be made clearer._

.github/workflows/build_opensearch.yaml

@@ -0,0 +1,36 @@
+---
+name: Build OpenSearch
+run-name: |
+  Build OpenSearch (attempt #${{ github.run_attempt }})
+
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: '0 0 2/2 * *' # https://crontab.guru/#0_0_2/2_*_*
+  push:
+    branches: [main]
+    tags: ['*']
+    paths:
+      # To check dependencies, run this ( you will need to consider transitive dependencies)
+      # bake --product PRODUCT -d | grep -v 'docker buildx bake' | jq '.target | keys[]'
+      - opensearch/**
+      - vector/**
+      - stackable-base/**
+      - stackable-devel/**
+      - jdk-base/**
+      - java-devel/**
+      - .github/actions/**
+      - .github/workflows/build_opensearch.yaml
+      - .github/workflows/reusable_build_image.yaml
+
+jobs:
+  build_image:
+    name: Reusable Workflow
+    uses: ./.github/workflows/reusable_build_image.yaml
+    secrets:
+      harbor-robot-secret: ${{ secrets.HARBOR_ROBOT_SDP_GITHUB_ACTION_BUILD_SECRET }}
+      slack-token: ${{ secrets.SLACK_CONTAINER_IMAGE_TOKEN }}
+    with:
+      product-name: opensearch
+      sdp-version: ${{ github.ref_type == 'tag' && github.ref_name || '0.0.0-dev' }}
+      registry-namespace: sdp

.github/workflows/preflight.yaml

@@ -66,6 +66,7 @@ jobs:
           - nifi
           - omid
           - opa
+          - opensearch
           - spark-k8s
           - superset
           - trino

.scripts/enumerate-product-versions.py

@@ -18,6 +18,7 @@
     "krb5",
     "nifi",
     "opa",
+    "opensearch",
     "omid",
     "spark-k8s",
     "superset",

.scripts/update_feature_tracker_db.sh

@@ -33,6 +33,7 @@ PRODUCT_CODE_NAMES=(
   hive
   kafka
   nifi
+  opensearch
   spark-k8s
   superset
   trino

CHANGELOG.md

@@ -7,12 +7,14 @@ All notable changes to this project will be documented in this file.
 ### Added
 
 - ubi9-rust-builder: Include `.tar.gz` snapshots of the operator source code in container images ([#1207])
+- opensearch: Add Opensearch as new product with version `3.1.0` ([#1215]).
 
 ### Changed
 
 - all: Use our build-repo to cache NPM dependencies ([#1219])
 
 [#1207]: https://github.com/stackabletech/docker-images/pull/1207
+[#1215]: https://github.com/stackabletech/docker-images/pull/1215
 [#1219]: https://github.com/stackabletech/docker-images/pull/1219
 
 ## [25.7.0] - 2025-07-23

README.md

@@ -8,9 +8,9 @@ This repository contains Dockerfiles and scripts to build base images for use wi
 | [![Build Airflow]][build_airflow.yaml] | [![Build Druid]][build_druid.yaml] | [![Build Hadoop]][build_hadoop.yaml] | [![Build HBase]][build_hbase.yaml] |
 | [![Build Hive]][build_hive.yaml] | [![Build Java Base]][build_java-base.yaml] | [![Build Java Development]][build_java-devel.yaml] | [![Build Kafka Testing Tools]][build_kafka-testing-tools.yaml] |
 | [![Build Kafka]][build_kafka.yaml] | [![Build Krb5]][build_krb5.yaml] | [![Build NiFi]][build_nifi.yaml] | [![Build Omid]][build_omid.yaml] |
-| [![Build OPA]][build_opa.yaml] | [![Build Spark Connect Client]][build_spark-connect-client.yaml] | [![Build Spark K8s]][build_spark-k8s.yaml] | [![Build Stackable Base]][build_stackable-base.yaml] |
-| [![Build Superset]][build_superset.yaml] | [![Build Testing Tools]][build_testing-tools.yaml] | [![Build Tools]][build_tools.yaml] | [![Build Trino CLI]][build_trino-cli.yaml] |
-| [![Build Trino]][build_trino.yaml] | [![Build Vector]][build_vector.yaml] | [![Build ZooKeeper]][build_zookeeper.yaml] | |
+| [![Build OPA]][build_opa.yaml] | [![Build OpenSearch]][build_opensearch.yaml] | [![Build Spark Connect Client]][build_spark-connect-client.yaml] | [![Build Spark K8s]][build_spark-k8s.yaml] |
+| [![Build Stackable Base]][build_stackable-base.yaml] | [![Build Superset]][build_superset.yaml] | [![Build Testing Tools]][build_testing-tools.yaml] | [![Build Tools]][build_tools.yaml] |
+| [![Build Trino CLI]][build_trino-cli.yaml] | [![Build Trino]][build_trino.yaml] | [![Build Vector]][build_vector.yaml] | [![Build ZooKeeper]][build_zookeeper.yaml] |
 <!-- end:badges -->
 
 ## Prerequisites
@@ -239,6 +239,8 @@ ENTRYPOINT ["/stackable-zookeeper-operator"]
 [build_omid.yaml]: https://github.com/stackabletech/docker-images/actions/workflows/build_omid.yaml
 [Build OPA]: https://github.com/stackabletech/docker-images/actions/workflows/build_opa.yaml/badge.svg
 [build_opa.yaml]: https://github.com/stackabletech/docker-images/actions/workflows/build_opa.yaml
+[Build OpenSearch]: https://github.com/stackabletech/docker-images/actions/workflows/build_opensearch.yaml/badge.svg
+[build_opensearch.yaml]: https://github.com/stackabletech/docker-images/actions/workflows/build_opensearch.yaml
 [Build Spark Connect Client]: https://github.com/stackabletech/docker-images/actions/workflows/build_spark-connect-client.yaml/badge.svg
 [build_spark-connect-client.yaml]: https://github.com/stackabletech/docker-images/actions/workflows/build_spark-connect-client.yaml
 [Build Spark K8s]: https://github.com/stackabletech/docker-images/actions/workflows/build_spark-k8s.yaml/badge.svg

conf.py

@@ -22,12 +22,17 @@
 hive = importlib.import_module("hive.versions")
 java_base = importlib.import_module("java-base.versions")
 java_devel = importlib.import_module("java-devel.versions")
+jdk_base = importlib.import_module("jdk-base.versions")
 kafka = importlib.import_module("kafka.versions")
 krb5 = importlib.import_module("krb5.versions")
 vector = importlib.import_module("vector.versions")
 nifi = importlib.import_module("nifi.versions")
 omid = importlib.import_module("omid.versions")
 opa = importlib.import_module("opa.versions")
+opensearch = importlib.import_module("opensearch.versions")
+opensearch_security_plugin = importlib.import_module(
+    "opensearch.security-plugin.versions"
+)
 spark_k8s = importlib.import_module("spark-k8s.versions")
 stackable_base = importlib.import_module("stackable-base.versions")
 stackable_devel = importlib.import_module("stackable-devel.versions")
@@ -58,12 +63,18 @@
     {"name": "hive", "versions": hive.versions},
     {"name": "java-base", "versions": java_base.versions},
     {"name": "java-devel", "versions": java_devel.versions},
+    {"name": "jdk-base", "versions": jdk_base.versions},
     {"name": "kafka", "versions": kafka.versions},
     {"name": "krb5", "versions": krb5.versions},
     {"name": "vector", "versions": vector.versions},
     {"name": "nifi", "versions": nifi.versions},
     {"name": "omid", "versions": omid.versions},
     {"name": "opa", "versions": opa.versions},
+    {"name": "opensearch", "versions": opensearch.versions},
+    {
+        "name": "opensearch/security-plugin",
+        "versions": opensearch_security_plugin.versions,
+    },
     {"name": "spark-k8s", "versions": spark_k8s.versions},
     {"name": "stackable-base", "versions": stackable_base.versions},
     {"name": "stackable-devel", "versions": stackable_devel.versions},
@@ -91,6 +102,7 @@
     "kafka": {"id": "625ff25b91bdcd4b49c823a4"},
     "nifi": {"id": "625586a32e9e14bc8118e203"},
     "opa": {"id": "6255838bea1feb8bec4aaaa3"},
+    "opensearch": {"id": "6880fe690db664aa303d3a28"},
     "spark-k8s": {"id": "62613e81f8ce82a2f247dda5"},
     "superset": {"id": "62557e5fea1feb8bec4aaaa0"},
     "tools": {"id": "62557cd575ab7e30884aaaa0"},

jdk-base/Dockerfile

@@ -0,0 +1,59 @@
+# syntax=docker/dockerfile:1.16.0@sha256:e2dd261f92e4b763d789984f6eab84be66ab4f5f08052316d8eb8f173593acf7
+# check=error=true
+
+#
+# Provides the common Java Development Kit for SDP products
+#
+FROM stackable/image/vector
+
+ARG PRODUCT
+ARG RELEASE="1"
+
+LABEL name="Stackable image for OpenJDK" \
+    maintainer="[email protected]" \
+    vendor="Stackable GmbH" \
+    version="${PRODUCT}" \
+    release="${RELEASE}" \
+    summary="The Stackable OpenJDK base image." \
+    description="This image is the base image for all Stackable Java product images which require a JDK."
+
+# See: https://adoptium.net/en-gb/installation/linux/#_centosrhelfedora_instructions
+RUN cat <<EOF > /etc/yum.repos.d/adoptium.repo
+[Adoptium]
+name=Adoptium
+# The adoptium mirror times-out often, so we have created a pull-through cache
+# baseurl=https://packages.adoptium.net/artifactory/rpm/rhel/\$releasever/\$basearch
+baseurl=https://build-repo.stackable.tech/repository/Adoptium/\$releasever/\$basearch
+enabled=1
+gpgcheck=1
+gpgkey=https://packages.adoptium.net/artifactory/api/gpg/key/public
+EOF
+
+RUN microdnf update && \
+    microdnf install \
+    # Needed to run Java programs
+    "temurin-${PRODUCT}-jdk" \
+    # Needed, because otherwise e.g. Zookeeper fails with
+    # Caused by: java.io.FileNotFoundException: /usr/lib/jvm/java-11-openjdk-11.0.20.0.8-2.el8.x86_64/lib/tzdb.dat (No such file or directory)
+    tzdata-java \
+    # Most of the Java tools needs at least "klist" and "kinit" for Kerberos integration
+    krb5-workstation \
+    --nodocs && \
+    microdnf clean all
+
+COPY java-base/licenses /licenses
+
+ENV JAVA_HOME="/usr/lib/jvm/temurin-${PRODUCT}-jdk"
+
+# This image doesn't include the development packages for Java.
+# For images that need the devel package (ex. Spark) use this env variable to
+# install the proper javac version.
+#
+# microdnf install java-${JAVA_VERSION}-openjdk-devel
+#
+ENV JAVA_VERSION=$PRODUCT
+
+# Mitigation for CVE-2021-44228 (Log4Shell)
+# This variable is supported as of Log4j version 2.10 and
+# disables the vulnerable feature
+ENV LOG4J_FORMAT_MSG_NO_LOOKUPS=true