-
-
Notifications
You must be signed in to change notification settings - Fork 6
Make uid/gid configurable & change group of files - part 2 #890
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
e7e3283
Make uid/gid configurable & change group of files
lfrancke e6606ef
More tools now migrated but not tested yet:
lfrancke 6d4f536
- OPA
lfrancke 8ad5fa4
Adds Spark and a changelog entry
lfrancke 72a2c58
Merge branch 'main' into feat/user-handling-pt2
lfrancke 85536f3
Merge branch 'main' into feat/user-handling-pt2
NickLarsenNZ e143131
Update CHANGELOG.md
lfrancke 4411137
Update comment
lfrancke File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8,6 +8,7 @@ FROM stackable/image/stackable-base AS final | |
| ARG PRODUCT | ||
| ARG KCAT | ||
| ARG RELEASE | ||
| ARG STACKABLE_USER_UID | ||
|
|
||
| LABEL name="Kafka Testing Tools" \ | ||
| maintainer="[email protected]" \ | ||
|
|
@@ -29,11 +30,10 @@ RUN microdnf install \ | |
| && rm -rf /var/cache/yum | ||
|
|
||
| # Store kcat version with binary name and add softlink | ||
| COPY --chown=stackable:stackable --from=kcat /stackable/kcat-${KCAT}/kcat /stackable/kcat-${KCAT} | ||
| COPY --chown=${STACKABLE_USER_UID}:0 --from=kcat /stackable/kcat-${KCAT}/kcat /stackable/kcat-${KCAT} | ||
| RUN ln -s /stackable/kcat-${KCAT} /stackable/kcat | ||
| COPY --chown=stackable:stackable --from=kcat /licenses /licenses | ||
| COPY --chown=${STACKABLE_USER_UID}:0 --from=kcat /licenses /licenses | ||
|
|
||
|
|
||
| COPY --chown=stackable:stackable kafka-testing-tools/licenses /licenses | ||
| COPY --chown=${STACKABLE_USER_UID}:0 kafka-testing-tools/licenses /licenses | ||
|
|
||
| ENTRYPOINT ["/stackable/kcat"] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,8 +9,9 @@ ARG PRODUCT | |
| ARG SCALA | ||
| ARG OPA_AUTHORIZER | ||
| ARG JMX_EXPORTER | ||
| ARG STACKABLE_USER_UID | ||
|
|
||
| USER stackable | ||
| USER ${STACKABLE_USER_UID} | ||
| WORKDIR /stackable | ||
|
|
||
| RUN curl "https://repo.stackable.tech/repository/packages/kafka/kafka-${PRODUCT}-src.tgz" | tar -xzC . && \ | ||
|
|
@@ -27,35 +28,20 @@ RUN curl "https://repo.stackable.tech/repository/packages/kafka/kafka-${PRODUCT} | |
| RUN curl https://repo.stackable.tech/repository/packages/kafka-opa-authorizer/opa-authorizer-${OPA_AUTHORIZER}-all.jar \ | ||
| -o /stackable/kafka_${SCALA}-${PRODUCT}/libs/opa-authorizer-${OPA_AUTHORIZER}-all.jar | ||
|
|
||
| COPY --chown=stackable:stackable kafka/stackable/jmx/ /stackable/jmx/ | ||
| COPY --chown=${STACKABLE_USER_UID}:0 kafka/stackable/jmx/ /stackable/jmx/ | ||
| RUN curl https://repo.stackable.tech/repository/packages/jmx-exporter/jmx_prometheus_javaagent-${JMX_EXPORTER}.jar \ | ||
| -o /stackable/jmx/jmx_prometheus_javaagent-${JMX_EXPORTER}.jar && \ | ||
| chmod +x /stackable/jmx/jmx_prometheus_javaagent-${JMX_EXPORTER}.jar && \ | ||
| ln -s /stackable/jmx/jmx_prometheus_javaagent-${JMX_EXPORTER}.jar /stackable/jmx/jmx_prometheus_javaagent.jar | ||
|
|
||
| # For earlier versions this script removes the .class file that contains the | ||
| # vulnerable code. | ||
| # TODO: This can be restricted to target only versions which do not honor the environment | ||
| # varible that has been set above but this has not currently been implemented | ||
| COPY shared/log4shell.sh /bin | ||
| RUN /bin/log4shell.sh /stackable/kafka_${SCALA}-${PRODUCT} | ||
|
|
||
| # Ensure no vulnerable files are left over | ||
| # This will currently report vulnerable files being present, as it also alerts on | ||
| # SocketNode.class, which we do not remove with our scripts. | ||
| # Further investigation will be needed whether this should also be removed. | ||
| COPY shared/log4shell_1.6.1-log4shell_Linux_x86_64 /bin/log4shell_scanner_x86_64 | ||
| COPY shared/log4shell_1.6.1-log4shell_Linux_aarch64 /bin/log4shell_scanner_aarch64 | ||
| COPY shared/log4shell_scanner /bin/log4shell_scanner | ||
| RUN /bin/log4shell_scanner s /stackable/kafka_${SCALA}-${PRODUCT} | ||
| # === | ||
|
|
||
| FROM stackable/image/java-base AS final | ||
|
|
||
| ARG RELEASE | ||
| ARG PRODUCT | ||
| ARG SCALA | ||
| ARG KCAT | ||
| ARG STACKABLE_USER_UID | ||
|
|
||
| LABEL name="Apache Kafka" \ | ||
| maintainer="[email protected]" \ | ||
|
|
@@ -67,32 +53,38 @@ LABEL name="Apache Kafka" \ | |
|
|
||
| # This is needed for kubectl | ||
| COPY kafka/kubernetes.repo /etc/yum.repos.d/kubernetes.repo | ||
| RUN microdnf update && \ | ||
| microdnf install \ | ||
| # needed by kcat for kerberos | ||
| cyrus-sasl-gssapi \ | ||
| # Can be removed once listener-operator integration is used | ||
| kubectl && \ | ||
| microdnf clean all && \ | ||
| rpm -qa --qf "%{NAME}-%{VERSION}-%{RELEASE}\n" | sort > /stackable/package_manifest.txt && \ | ||
| rm -rf /var/cache/yum | ||
|
|
||
| USER stackable | ||
| WORKDIR /stackable | ||
|
|
||
| COPY --chown=stackable:stackable kafka/licenses /licenses | ||
| COPY --chown=${STACKABLE_USER_UID}:0 kafka/licenses /licenses | ||
| COPY --chown=${STACKABLE_USER_UID}:0 --from=kafka-builder /stackable/kafka_${SCALA}-${PRODUCT} /stackable/kafka_${SCALA}-${PRODUCT} | ||
| COPY --chown=${STACKABLE_USER_UID}:0 --from=kafka-builder /stackable/jmx/ /stackable/jmx/ | ||
| COPY --chown=${STACKABLE_USER_UID}:0 --from=kcat /stackable/kcat-${KCAT}/kcat /stackable/bin/kcat-${KCAT} | ||
| COPY --chown=${STACKABLE_USER_UID}:0 --from=kcat /licenses /licenses | ||
|
|
||
| # We copy opa-authorizer.jar and jmx-exporter through the builder image to have an absolutely minimal final image | ||
| # (e.g. we don't even need curl in it). | ||
| COPY --chown=stackable:stackable --from=kafka-builder /stackable/kafka_${SCALA}-${PRODUCT} /stackable/kafka_${SCALA}-${PRODUCT} | ||
| COPY --chown=stackable:stackable --from=kafka-builder /stackable/jmx/ /stackable/jmx/ | ||
| COPY --chown=stackable:stackable --from=kcat /stackable/kcat-${KCAT}/kcat /stackable/bin/kcat-${KCAT} | ||
| COPY --chown=stackable:stackable --from=kcat /licenses /licenses | ||
| WORKDIR /stackable | ||
|
|
||
| RUN ln -s /stackable/bin/kcat-${KCAT} /stackable/bin/kcat && \ | ||
| # kcat was located in /stackable/kcat - legacy | ||
| ln -s /stackable/bin/kcat /stackable/kcat && \ | ||
| ln -s /stackable/kafka_${SCALA}-${PRODUCT} /stackable/kafka | ||
| RUN <<EOF | ||
| microdnf update | ||
| # cyrus-sasl-gssapi: needed by kcat for kerberos | ||
| # kubectl: Can be removed once listener-operator integration is used | ||
| microdnf install \ | ||
| cyrus-sasl-gssapi \ | ||
| kubectl | ||
|
|
||
| microdnf clean all | ||
| rpm -qa --qf "%{NAME}-%{VERSION}-%{RELEASE}\n" | sort > /stackable/package_manifest.txt | ||
| rm -rf /var/cache/yum | ||
|
|
||
| ln -s /stackable/bin/kcat-${KCAT} /stackable/bin/kcat | ||
| # kcat was located in /stackable/kcat - legacy | ||
| ln -s /stackable/bin/kcat /stackable/kcat | ||
| ln -s /stackable/kafka_${SCALA}-${PRODUCT} /stackable/kafka | ||
|
|
||
| # All files and folders owned by root to support running as arbitrary users | ||
| # This is best practice as all container users will belong to the root group (0) | ||
| chown -R ${STACKABLE_USER_UID}:0 /stackable | ||
| chmod -R g=u /stackable | ||
| EOF | ||
|
|
||
| USER ${STACKABLE_USER_UID} | ||
|
|
||
| ENV PATH="${PATH}:/stackable/bin:/stackable/kafka/bin" | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,6 +5,7 @@ FROM stackable/image/java-devel AS nifi-builder | |
|
|
||
| ARG PRODUCT | ||
| ARG MAVEN_VERSION="3.9.8" | ||
| ARG STACKABLE_USER_UID | ||
|
|
||
| RUN microdnf update && \ | ||
| microdnf clean all && \ | ||
|
|
@@ -22,10 +23,10 @@ RUN if [[ "${PRODUCT}" == 2.* ]] ; then \ | |
| ln -sf /tmp/apache-maven-${MAVEN_VERSION}/bin/mvn /usr/bin/mvn ; \ | ||
| fi | ||
|
|
||
| USER stackable | ||
| USER ${STACKABLE_USER_UID} | ||
| WORKDIR /stackable | ||
|
|
||
| COPY --chown=stackable:stackable nifi/stackable/patches /stackable/patches | ||
| COPY --chown=${STACKABLE_USER_UID}:0 nifi/stackable/patches /stackable/patches | ||
|
|
||
| # NOTE: NiFi 1.21.0 source build does not work with the current arm64 git runners due to java heap issues: | ||
| # | ||
|
|
@@ -82,28 +83,11 @@ RUN if [[ "${PRODUCT}" == "1.21.0" ]] ; then \ | |
| rm -rf /stackable/nifi-${PRODUCT}/docs ; \ | ||
| fi | ||
|
|
||
| # === | ||
| # For earlier versions this script removes the .class file that contains the | ||
| # vulnerable code. | ||
| # TODO: This can be restricted to target only versions which do not honor the environment | ||
| # varible that has been set above but this has not currently been implemented | ||
| COPY shared/log4shell.sh /bin | ||
| RUN /bin/log4shell.sh /stackable/nifi-${PRODUCT} | ||
|
|
||
| # Ensure no vulnerable files are left over | ||
| # This will currently report vulnerable files being present, as it also alerts on | ||
| # SocketNode.class, which we do not remove with our scripts. | ||
| # Further investigation will be needed whether this should also be removed. | ||
| COPY shared/log4shell_1.6.1-log4shell_Linux_x86_64 /bin/log4shell_scanner_x86_64 | ||
| COPY shared/log4shell_1.6.1-log4shell_Linux_aarch64 /bin/log4shell_scanner_aarch64 | ||
| COPY shared/log4shell_scanner /bin/log4shell_scanner | ||
| RUN /bin/log4shell_scanner s /stackable/nifi-${PRODUCT} | ||
| # === | ||
|
|
||
| FROM stackable/image/java-base AS final | ||
|
|
||
| ARG PRODUCT | ||
| ARG RELEASE | ||
| ARG STACKABLE_USER_UID | ||
|
|
||
| LABEL name="Apache NiFi" \ | ||
| maintainer="[email protected]" \ | ||
|
|
@@ -113,28 +97,39 @@ LABEL name="Apache NiFi" \ | |
| summary="The Stackable image for Apache NiFi." \ | ||
| description="This image is deployed by the Stackable Operator for Apache NiFi." | ||
|
|
||
| RUN microdnf update && \ | ||
| microdnf install \ | ||
| # Required to install nipyapi | ||
| python-pip && \ | ||
| microdnf clean all && \ | ||
| rm -rf /var/cache/yum && \ | ||
| # The nipyapi is required for the ReportingTaskJob | ||
| pip install --no-cache-dir nipyapi==0.19.1 && \ | ||
| # For backwards compatibility we create a softlink in /bin where the jar used to be as long as we are root | ||
| # This can be removed once older versions / operators using this are no longer supported | ||
| ln -s /stackable/stackable-bcrypt.jar /bin/stackable-bcrypt.jar | ||
| COPY --chown=${STACKABLE_USER_UID}:0 --from=nifi-builder /stackable/nifi-${PRODUCT} /stackable/nifi-${PRODUCT}/ | ||
| COPY --chown=${STACKABLE_USER_UID}:0 --from=nifi-builder /stackable/stackable-bcrypt.jar /stackable/stackable-bcrypt.jar | ||
|
|
||
| COPY --chown=${STACKABLE_USER_UID}:0 nifi/stackable/bin /stackable/bin | ||
| COPY --chown=${STACKABLE_USER_UID}:0 nifi/licenses /licenses | ||
| COPY --chown=${STACKABLE_USER_UID}:0 nifi/python /stackable/python | ||
|
|
||
| RUN <<EOF | ||
| ln -s /stackable/nifi-${PRODUCT} /stackable/nifi | ||
|
|
||
| microdnf update | ||
|
|
||
| # python-pip: Required to install nipyapi | ||
| microdnf install \ | ||
| python-pip | ||
|
|
||
| microdnf clean all | ||
| rm -rf /var/cache/yum | ||
|
|
||
| USER stackable | ||
| # The nipyapi is required for the ReportingTaskJob | ||
| pip install --no-cache-dir nipyapi==0.19.1 && \ | ||
|
|
||
| COPY --chown=stackable:stackable --from=nifi-builder /stackable/nifi-${PRODUCT} /stackable/nifi-${PRODUCT}/ | ||
| COPY --chown=stackable:stackable --from=nifi-builder /stackable/stackable-bcrypt.jar /stackable/stackable-bcrypt.jar | ||
| # For backwards compatibility we create a softlink in /bin where the jar used to be as long as we are root | ||
| # This can be removed once older versions / operators using this are no longer supported | ||
| ln -s /stackable/stackable-bcrypt.jar /bin/stackable-bcrypt.jar | ||
|
|
||
| COPY --chown=stackable:stackable nifi/stackable/bin /stackable/bin | ||
| COPY --chown=stackable:stackable nifi/licenses /licenses | ||
| COPY --chown=stackable:stackable nifi/python /stackable/python | ||
| # All files and folders owned by root to support running as arbitrary users | ||
| # This is best practice as all container users will belong to the root group (0) | ||
| chown -R ${STACKABLE_USER_UID}:0 /stackable | ||
| chmod -R g=u /stackable | ||
| EOF | ||
|
|
||
| RUN ln -s /stackable/nifi-${PRODUCT} /stackable/nifi | ||
| USER ${STACKABLE_USER_UID} | ||
|
|
||
| ENV HOME=/stackable | ||
| ENV NIFI_HOME=/stackable/nifi | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.