feat(docker): Create non-root `clp-user`; Move `USER` and `ENV` directives after flattening (fixes #1379); Reduce layers using multi-line ENV (fixes #1378). #1413

junhaoliao · 2025-10-13T02:48:50Z

To fix #1410, we need to "Move USER and ENV directives after image flattening in clp-package Dockerfile"

junhaoliao · 2025-10-30T06:37:23Z

the user home path setting here broke the s3 profile auth type because the CONTAINER_AWS_CONFIG_DIRECTORY's value should be relative to the CLP_HOME but isn't

@coderabbitai create an issue to track this

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash # Create a GitHub issue to track the S3 profile auth type breakage gh issue create \ --repo y-scope/clp \ --title "S3 profile auth type broken due to user home path change in Docker image" \ --body "## Problem The user home path setting introduced in PR #1413 broke the S3 profile auth type because \`CONTAINER_AWS_CONFIG_DIRECTORY\`'s value should be relative to \`CLP_HOME\` but isn't. ## Context - PR #1413 sets the home directory for \`clp-user\` to \`CLP_HOME\` (\`/opt/clp\`) in the Docker image - \`CONTAINER_AWS_CONFIG_DIRECTORY\` is currently hardcoded to \`/.aws\` - This mismatch breaks S3 profile authentication ## Related - PR: https://github.com/y-scope/clp/pull/1413 - Comment: https://github.com/y-scope/clp/pull/1413#discussion_r2476592326 - Reported by: @junhaoliao" \ --assignee junhaoliao

Length of output: 42

@junhaoliao, I've created issue #1524 to track the S3 profile auth type breakage caused by the user home path change. The issue documents how the CONTAINER_AWS_CONFIG_DIRECTORY hardcoded path conflicts with the new home directory setting.

#1524

🧠 Learnings used

Learnt from: junhaoliao PR: y-scope/clp#1414 File: tools/docker-images/clp-package/Dockerfile:20-24 Timestamp: 2025-10-13T03:32:19.293Z Learning: In the clp repository's Dockerfiles (e.g., tools/docker-images/clp-package/Dockerfile), ENV directives should be split into separate lines for readability rather than consolidated to reduce layer count. This is especially true for PATH modifications, as agreed upon in PR #1166. Later ENV settings may depend on earlier ones (e.g., referencing CLP_HOME).

Learnt from: junhaoliao PR: y-scope/clp#1413 File: tools/docker-images/clp-package/Dockerfile:22-24 Timestamp: 2025-10-20T21:05:30.417Z Learning: In the clp repository's Dockerfiles, ENV directives should be consolidated into multi-line ENV statements when possible to reduce image layers. ENV statements should only be split into separate commands when consolidation is not possible due to dependencies (e.g., when later variables must reference earlier ones that need to be set first, or when PATH must be modified sequentially).

Learnt from: junhaoliao PR: y-scope/clp#1178 File: components/clp-package-utils/clp_package_utils/controller.py:217-223 Timestamp: 2025-09-25T05:13:13.298Z Learning: The compression scheduler service in CLP runs with CLP_UID_GID (current user's UID:GID) rather than CLP_SERVICE_CONTAINER_UID_GID (999:999), unlike infrastructure services such as database, queue, redis, and results cache which run with the service container UID:GID.

Learnt from: junhaoliao PR: y-scope/clp#1335 File: taskfiles/docker-images.yaml:15-15 Timestamp: 2025-09-25T19:26:32.436Z Learning: In the CLP project's Taskfile, the `:package` task creates the G_PACKAGE_BUILD_DIR directory structure, so any task that depends on `:package` (like `docker-images:package`) can safely assume this directory exists without needing additional mkdir commands.

Learnt from: jackluo923 PR: y-scope/clp#1054 File: components/core/tools/docker-images/clp-env-base-musllinux_1_2-aarch64/build.sh:3-5 Timestamp: 2025-07-07T17:43:04.349Z Learning: In CLP project build scripts (specifically build.sh files in docker-images directories), maintain consistency with the established pattern of using separate `set -eu` and `set -o pipefail` commands rather than combining them into `set -euo pipefail`, to ensure uniform script structure across all platform build scripts.

Learnt from: jackluo923 PR: y-scope/clp#1054 File: components/core/tools/docker-images/clp-env-base-musllinux_1_2-x86/build.sh:18-24 Timestamp: 2025-07-01T14:52:02.418Z Learning: In the CLP project, consistency across platform build scripts is prioritized over defensive programming when it comes to git remote handling. All build.sh files in docker-images directories should follow the same pattern for git metadata injection.

Learnt from: jackluo923 PR: y-scope/clp#1054 File: components/core/tools/docker-images/clp-env-base-musllinux_1_2-x86/build.sh:18-24 Timestamp: 2025-07-01T14:52:02.418Z Learning: In the CLP project, consistency across platform build scripts is prioritized over defensive programming when it comes to git remote handling. All build.sh files in docker-images directories should follow the same pattern for git metadata injection.

Learnt from: junhaoliao PR: y-scope/clp#1178 File: components/clp-py-utils/clp_py_utils/clp_config.py:47-47 Timestamp: 2025-10-07T07:54:32.427Z Learning: In components/clp-py-utils/clp_py_utils/clp_config.py, the CONTAINER_AWS_CONFIG_DIRECTORY constant is intentionally set to pathlib.Path("/") / ".aws" (i.e., `/.aws`) rather than a user-specific home directory. This hardcoded path is part of the container orchestration design.

Learnt from: Bill-hbrhbr PR: y-scope/clp#1122 File: components/core/src/clp/clp/CMakeLists.txt:175-195 Timestamp: 2025-07-23T09:54:45.185Z Learning: In the CLP project, when reviewing CMakeLists.txt changes that introduce new compression library dependencies (BZip2, LibLZMA, LZ4, ZLIB), the team prefers to address conditional linking improvements in separate PRs rather than expanding the scope of focused migration PRs like the LibArchive task-based installation migration.

junhaoliao · 2025-10-20T21:07:20Z

Although Docker's documentation states that --chown= supports usernames and performs lookups in /etc/passwd and /etc/group within the container to resolve UID/GID, we must use ${UID} instead of ${USER} here because such lookups are not yet supported when --link is used.

This limitation is a known issue in Docker Buildx, tracked at moby/buildkit#2987

-Original file line number
+Diff line change
@@ Expand Up / @@ -12,15 +12,18 @@ RUN ./setup-scripts/install-prebuilt-packages.sh \ @@
     RUN apt-get clean \
         && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
-    ENV CLP_HOME="/opt/clp"
-    ENV PATH="${CLP_HOME}/bin:${PATH}"
-    ENV PATH="${CLP_HOME}/sbin:${PATH}"
-    ENV PYTHONPATH="${CLP_HOME}/lib/python3/site-packages"
-    USER 1000:1000
-    COPY --link ./build/clp-package /opt/clp
     # Flatten the image
     FROM scratch
     COPY --link --from=base / /
+    ARG UID=1000
+    ENV CLP_HOME="/opt/clp"
+    ENV PATH="${CLP_HOME}/sbin:${CLP_HOME}/bin:${PATH}" \
+        PYTHONPATH="${CLP_HOME}/lib/python3/site-packages" \
+        USER="clp-user"
+    RUN useradd --uid ${UID} --shell /bin/bash --home-dir ${CLP_HOME} ${USER}
+    USER ${USER}
+    WORKDIR ${CLP_HOME}
+    COPY --link --chown=${UID} ./build/clp-package ${CLP_HOME}

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(docker): Create non-root `clp-user`; Move `USER` and `ENV` directives after flattening (fixes #1379); Reduce layers using multi-line ENV (fixes #1378). #1413

Uh oh!

Diff view

Diff view

There are no files selected for viewing

junhaoliao Oct 13, 2025

Uh oh!

junhaoliao Oct 30, 2025

Uh oh!

coderabbitai bot Oct 30, 2025

Uh oh!

junhaoliao Oct 20, 2025

Uh oh!

Uh oh!

feat(docker): Create non-root clp-user; Move USER and ENV directives after flattening (fixes #1379); Reduce layers using multi-line ENV (fixes #1378). #1413

Uh oh!

feat(docker): Create non-root clp-user; Move USER and ENV directives after flattening (fixes #1379); Reduce layers using multi-line ENV (fixes #1378). #1413

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

junhaoliao Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

junhaoliao Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

junhaoliao Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

feat(docker): Create non-root `clp-user`; Move `USER` and `ENV` directives after flattening (fixes #1379); Reduce layers using multi-line ENV (fixes #1378). #1413

feat(docker): Create non-root `clp-user`; Move `USER` and `ENV` directives after flattening (fixes #1379); Reduce layers using multi-line ENV (fixes #1378). #1413