Skip to content

Commit 07d838e

Browse files
committed
address feedback in PR
1 parent 9162b57 commit 07d838e

File tree

10 files changed

+147
-11
lines changed

10 files changed

+147
-11
lines changed

.github/ISSUE_TEMPLATE/early-pre-release.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ Part of stackabletech/issues#xxx.
4545
- [ ] [Create issue from template: update-product-kafka.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-kafka.md)
4646
- [ ] [Create issue from template: update-product-nifi.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-nifi.md)
4747
- [ ] [Create issue from template: update-product-opa.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-opa.md)
48+
- [ ] [Create issue from template: update-product-opensearch.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-opensearch.md)
4849
- [ ] [Create issue from template: update-product-spark.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-spark.md)
4950
- [ ] [Create issue from template: update-product-superset.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-superset.md)
5051
- [ ] [Create issue from template: update-product-trino.md](https://github.com/stackabletech/docker-images/issues/new?template=update-product-trino.md)
Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
---
2+
name: Update OpenSearch
3+
about: >-
4+
This template contains instructions specific to updating this product and/or
5+
container image(s).
6+
title: >-
7+
chore(opensearch): Update container images ahead of Stackable Release YY.M.X
8+
labels: []
9+
# Currently, projects cannot be assigned via front-matter.
10+
projects: ['stackabletech/10']
11+
assignees: ''
12+
---
13+
14+
Part of #xxx.
15+
16+
<!--
17+
This gives hints to the person doing the work.
18+
Add/Change/Remove anything that isn't applicable anymore
19+
-->
20+
- Add: `x.x.x`
21+
- Remove: `y.y.y`
22+
23+
> [!TIP]
24+
> Please add the `scheduled-for/20XX-XX` label, and add to the [Stackable Engineering][1] project.
25+
>
26+
> [1]: https://github.com/orgs/stackabletech/projects/10
27+
28+
## Update tasks
29+
30+
- [ ] Update `versions.py` to reflect the agreed upon versions in the spreadsheet (including the removal of old versions).
31+
- [ ] Update `versions.py` to the latest supported version of JVM (base and devel).
32+
- [ ] Update other dependencies if applicable (eg: security-plugin, etc).
33+
- [ ] Check other operators (getting_started / kuttl / supported-versions) for usage of the versions. Add the PR(s) to the list below.
34+
- [ ] Update the version in demos. Add the PR(s) to the list below.
35+
36+
## Related Pull Requests
37+
38+
> [!TIP]
39+
> Delete any items that do not apply so that all applicable items can be checked.
40+
> For example, if you add release notes to the documentation repository, you do not need the latter two criteria.
41+
42+
- _Link to the docker-images PR (product update)_
43+
- _Link to the operator PR (getting_started / kuttl / supported-versions)_
44+
- _Link to any other operator PRs (getting_started / kuttl)_
45+
- _Link to demo PR (raise against the `main` branch)_
46+
- _Link to the Release Notes PR in the documentation repo (if not a comment below)_
47+
48+
## Acceptance
49+
50+
> [!TIP]
51+
> This list should be completed by the assignee(s), once respective PRs have been merged. Once all items have been
52+
> checked, the issue can be moved into _Development: Done_.
53+
54+
- [ ] Can build image (either locally, or in CI)
55+
- [ ] Kuttl smoke tests passes (either locally, or in CI)
56+
- [ ] Release notes added to documentation and linked as a PR above
57+
- [ ] Release notes written in a comment below
58+
- [ ] Applicable `release-note` label added to this issue
59+
60+
<details>
61+
<summary>Testing instructions</summary>
62+
63+
```shell
64+
# See the latest version at https://pypi.org/project/image-tools-stackabletech/
65+
pip install image-tools-stackabletech==0.0.16
66+
67+
bake --product opensearch=x.y.z # where x.y.z is the new version added in this PR
68+
69+
kind load docker-image oci.stackable.tech/sdp/opensearch:x.y.z-stackable0.0.0-dev
70+
71+
# Change directory into the opa-operator repository and update the
72+
# product version in tests/test-definition.yaml
73+
./scripts/run-tests --test-suite smoke-latest # or similar
74+
```
75+
76+
</details>
77+
78+
_Please consider updating this template if these instructions are wrong, or
79+
could be made clearer._
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
---
2+
name: Build OpenSearch
3+
run-name: |
4+
Build OpenSearch (attempt #${{ github.run_attempt }})
5+
6+
on:
7+
workflow_dispatch:
8+
schedule:
9+
- cron: '0 0 2/2 * *' # https://crontab.guru/#0_0_2/2_*_*
10+
push:
11+
branches: [main]
12+
tags: ['*']
13+
paths:
14+
# To check dependencies, run this ( you will need to consider transitive dependencies)
15+
# bake --product PRODUCT -d | grep -v 'docker buildx bake' | jq '.target | keys[]'
16+
- opensearch/**
17+
- vector/**
18+
- stackable-base/**
19+
- java-base/**
20+
- java-devel/**
21+
- .github/actions/**
22+
- .github/workflows/build_opensearch.yaml
23+
- .github/workflows/reusable_build_image.yaml
24+
25+
jobs:
26+
build_image:
27+
name: Reusable Workflow
28+
uses: ./.github/workflows/reusable_build_image.yaml
29+
secrets:
30+
harbor-robot-secret: ${{ secrets.HARBOR_ROBOT_SDP_GITHUB_ACTION_BUILD_SECRET }}
31+
slack-token: ${{ secrets.SLACK_CONTAINER_IMAGE_TOKEN }}
32+
with:
33+
product-name: kafka
34+
sdp-version: ${{ github.ref_type == 'tag' && github.ref_name || '0.0.0-dev' }}
35+
registry-namespace: sdp

.github/workflows/preflight.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@ jobs:
6666
- nifi
6767
- omid
6868
- opa
69+
- opensearch
6970
- spark-k8s
7071
- superset
7172
- trino

.scripts/enumerate-product-versions.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
"krb5",
1919
"nifi",
2020
"opa",
21+
"opensearch",
2122
"omid",
2223
"spark-k8s",
2324
"superset",

.scripts/update_feature_tracker_db.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ PRODUCT_CODE_NAMES=(
3333
hive
3434
kafka
3535
nifi
36+
opensearch
3637
spark-k8s
3738
superset
3839
trino

README.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,9 @@ This repository contains Dockerfiles and scripts to build base images for use wi
88
| [![Build Airflow]][build_airflow.yaml] | [![Build Druid]][build_druid.yaml] | [![Build Hadoop]][build_hadoop.yaml] | [![Build HBase]][build_hbase.yaml] |
99
| [![Build Hive]][build_hive.yaml] | [![Build Java Base]][build_java-base.yaml] | [![Build Java Development]][build_java-devel.yaml] | [![Build Kafka Testing Tools]][build_kafka-testing-tools.yaml] |
1010
| [![Build Kafka]][build_kafka.yaml] | [![Build Krb5]][build_krb5.yaml] | [![Build NiFi]][build_nifi.yaml] | [![Build Omid]][build_omid.yaml] |
11-
| [![Build OPA]][build_opa.yaml] | [![Build Spark Connect Client]][build_spark-connect-client.yaml] | [![Build Spark K8s]][build_spark-k8s.yaml] | [![Build Stackable Base]][build_stackable-base.yaml] |
12-
| [![Build Superset]][build_superset.yaml] | [![Build Testing Tools]][build_testing-tools.yaml] | [![Build Tools]][build_tools.yaml] | [![Build Trino CLI]][build_trino-cli.yaml] |
13-
| [![Build Trino]][build_trino.yaml] | [![Build Vector]][build_vector.yaml] | [![Build ZooKeeper]][build_zookeeper.yaml] | |
11+
| [![Build OPA]][build_opa.yaml] | [![Build OpenSearch]][build_opensearch.yaml] | [![Build Spark Connect Client]][build_spark-connect-client.yaml] | [![Build Spark K8s]][build_spark-k8s.yaml] |
12+
| [![Build Stackable Base]][build_stackable-base.yaml] | [![Build Superset]][build_superset.yaml] | [![Build Testing Tools]][build_testing-tools.yaml] | [![Build Tools]][build_tools.yaml] |
13+
| [![Build Trino CLI]][build_trino-cli.yaml] | [![Build Trino]][build_trino.yaml] | [![Build Vector]][build_vector.yaml] | [![Build ZooKeeper]][build_zookeeper.yaml] |
1414
<!-- end:badges -->
1515

1616
## Prerequisites
@@ -239,6 +239,8 @@ ENTRYPOINT ["/stackable-zookeeper-operator"]
239239
[build_omid.yaml]: https://github.com/stackabletech/docker-images/actions/workflows/build_omid.yaml
240240
[Build OPA]: https://github.com/stackabletech/docker-images/actions/workflows/build_opa.yaml/badge.svg
241241
[build_opa.yaml]: https://github.com/stackabletech/docker-images/actions/workflows/build_opa.yaml
242+
[Build OpenSearch]: https://github.com/stackabletech/docker-images/actions/workflows/build_opensearch.yaml/badge.svg
243+
[build_opensearch.yaml]: https://github.com/stackabletech/docker-images/actions/workflows/build_opensearch.yaml
242244
[Build Spark Connect Client]: https://github.com/stackabletech/docker-images/actions/workflows/build_spark-connect-client.yaml/badge.svg
243245
[build_spark-connect-client.yaml]: https://github.com/stackabletech/docker-images/actions/workflows/build_spark-connect-client.yaml
244246
[Build Spark K8s]: https://github.com/stackabletech/docker-images/actions/workflows/build_spark-k8s.yaml/badge.svg

opensearch/Dockerfile

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ NEW_VERSION="${PRODUCT}-stackable${RELEASE}"
3232
# Create snapshot of the source code including custom patches
3333
tar -czf /stackable/opensearch-${NEW_VERSION}-src.tar.gz .
3434
./scripts/build.sh -v "${PRODUCT}" -s false -a "${ARCH}"
35-
tar -xzf artifacts/dist/opensearch-min-${PRODUCT}-linux-${ARCH}.tar.gz -C /stackable
35+
tar -xzf "artifacts/dist/opensearch-min-${PRODUCT}-linux-${ARCH}.tar.gz" -C /stackable
3636
unzip artifacts/core-plugins/repository-s3-${PRODUCT}.zip -d /stackable/opensearch-${PRODUCT}/plugins/repository-s3/
3737
mv /stackable/opensearch-${PRODUCT}/plugins/repository-s3/config /stackable/opensearch-${PRODUCT}/config/repository-s3
3838
unzip artifacts/core-plugins/telemetry-otel-${PRODUCT}.zip -d /stackable/opensearch-${PRODUCT}/plugins/telemetry-otel/
@@ -65,14 +65,14 @@ rm -r jdk
6565
# Change the group permissions already in the builder image to reduce
6666
# the size of the final image.
6767
# see https://github.com/stackabletech/docker-images/issues/961
68-
chmod -R g=u "${HOME}"
6968
chmod +x /stackable/opensearch-${PRODUCT}/opensearch-docker-entrypoint.sh
69+
chmod -R g=u /stackable
7070
EOF
7171

7272
# The OpenSearch Performance Analyzer needs a JDK, not just a JRE.
7373
# With a JRE, the following exception is thrown:
7474
# java.lang.ClassNotFoundException: com.sun.tools.attach.VirtualMachine
75-
FROM stackable/image/jdk-base
75+
FROM stackable/image/jdk-base AS final
7676

7777
ARG PRODUCT
7878
ARG RELEASE
@@ -129,18 +129,22 @@ COPY \
129129
--from=opensearch-security-plugin \
130130
/stackable/src/opensearch/security-plugin/patchable-work/worktree/${OPENSEARCH_SECURITY_PLUGIN}/build/reports/bom.json \
131131
/stackable/opensearch-security-${OPENSEARCH_SECURITY_PLUGIN}-stackable${RELEASE}.cdx.json
132+
132133
RUN <<EOF
133134
microdnf update
134135
microdnf clean all
135136
rm -rf /var/cache/yum
136137

137138
# All files and folders owned by root group to support running as arbitrary users.
138139
# This is best practice as all container users will belong to the root group (0).
139-
chown -R ${STACKABLE_USER_UID}:0 ${HOME}
140-
chmod -R g=u ${HOME}
140+
chown ${STACKABLE_USER_UID}:0 ${HOME}
141+
chmod g=u /stackable/opensearch-${PRODUCT}-stackable${RELEASE}
142+
chmod g=u /stackable/opensearch-${PRODUCT}-stackable${RELEASE}/plugins/opensearch-security
143+
chmod g=u /stackable/*-src.tar.gz
144+
chmod g=u /stackable/*.cdx.json
141145
ln -s /stackable/opensearch-${PRODUCT}-stackable${RELEASE} ${OPENSEARCH_HOME}
142146
chown -h ${STACKABLE_USER_UID}:0 ${OPENSEARCH_HOME}
143-
EOF
147+
144148

145149
# ----------------------------------------
146150
# Checks
@@ -151,7 +155,7 @@ EOF
151155

152156
# Check that permissions and ownership in /stackable are set correctly
153157
# This will fail and stop the build if any mismatches are found.
154-
RUN <<EOF
158+
155159
/bin/check-permissions-ownership.sh /stackable ${STACKABLE_USER_UID} 0
156160
EOF
157161

opensearch/README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# OpenSearch
2+
3+
This is the Stackable Data Platform (SDP) OCI image for [OpenSearch](https://github.com/opensearch-project/opensearch).
4+
5+
The image includes:
6+
7+
- OpenSearch
8+
- `security` plugin
9+
- `telemetry-otel` plugin
10+
- `repository-s3` plugin
11+
- Source code as tarball
12+
- SBOMs in CycloneDX format

opensearch/security-plugin/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,5 +25,5 @@ EOF
2525
RUN <<EOF
2626
# Change the group permissions already in the builder image to reduce the size of the final image.
2727
# see https://github.com/stackabletech/docker-images/issues/961
28-
chmod -R g=u "${HOME}"
28+
chmod -R g=u /stackable
2929
EOF

0 commit comments

Comments
 (0)