Various fixes to container image and CI (#254)

## Summary

<!--
Include a short paragraph of the changes introduced in this PR.
If this PR requires additional context or rationale, explain why
the changes are necessary.
-->

Various fixes to containers images and CI. Containers images should now
reuse most layers on build, need less hacks when running in OCP, and
properly detect the guidellm version. Release and RC git tags/branches
tag the container image with the version. The `latest` and `stable` tags
now smartly update based on highest version tag (prev most recent
build).

## Details

<!--
Provide a detailed list of all changes introduced in this pull request.
-->
- Moves Containerfile to top-level
- Adds a `.containerignore` that mirrors `.gitignore`
- Fixes version detection in image builds
- Correctly sets release type and tags rc/release with version when
building images
- Use PDM to lock venv in image builds
- Adds weekly job for tagging latest and stable based on existing
version tagged images
- Adds weekly job to prune dev containers that are more than 2 weeks old
- Note: This job is currently set to dry-run and will be enabled in a
future PR
- Improves container compatibility with K8s unpriv users
- Reorders image layers to improve build caching

## Test Plan

<!--
List the steps needed to test this PR.
-->

### 1. Workflows 

- Workflows were partially tested with
[act](https://github.com/nektos/act) but need to be run on GitHub to
verify

### 2. Container Changes 

1. Run `podman run --rm ghcr.io/vllm-project/guidellm:pr-254 --version`
to verify version is set
2. Run `podman run --rm --entrypoint /opt/app-root/guidellm/bin/pip
ghcr.io/vllm-project/guidellm:pr-254 freeze` and verify that versions
match the `pylock.toml`.
3. Run a pod with the GuideLLM container and verify the user has write
permissions to `/home/guidellm`. (For tokenizer/dataset caching)

## Related Issues

<!--
Link any relevant issues that this PR addresses.
-->
- Resolves #

---

- [x] "I certify that all code in this PR is my own, except as noted
below."

## Use of AI

- [ ] Includes AI-assisted code completion
- [ ] Includes code generated by an AI application
- [ ] Includes AI-generated tests (NOTE: AI written tests should have a
docstring that includes `## WRITTEN BY AI ##`)

---------

Signed-off-by: Samuel Monson <[email protected]>

Author: sjmonson

.containerignore

@@ -0,0 +1 @@
+.gitignore

.github/workflows/container-maintenance.yml

@@ -0,0 +1,49 @@
+name: Container Image Maintenance
+
+on:
+  schedule:
+    - cron: '0 2 * * 3'  # Runs at 2am on Wednesdays
+  workflow_dispatch:  # Enables manual triggering of the workflow
+
+# Only run one at a time
+concurrency:
+  group: ${{ github.workflow }}
+
+jobs:
+  cleanup-container-tags:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Delete PR and untagged images older than 2 weeks
+        uses: snok/[email protected]
+        with:
+          account: ${{ github.actor }}
+          token: ${{ github.token }}
+          image-names: ${{ github.event.repository.name }}
+          image-tags: "pr-*"
+          cut-off: 2w
+          dry-run: true
+
+  push-container-tags:
+    runs-on: ubuntu-latest
+    needs: cleanup-container-tags
+    if: always() # Run after cleanup even if it fails
+    steps:
+      - name: Log into ghcr.io
+        uses: redhat-actions/podman-login@v1
+        with:
+          username: ${{ github.actor }}
+          password: ${{ github.token }}
+          registry: ghcr.io/${{ github.repository_owner }}
+      - name: Get list of tags
+        run: |
+          skopeo list-tags docker://${{ github.repository }} | jq --raw-output '.Tags[]' > tags
+      - name: Get latest release and rc tags
+        run: |
+          STABLE_TAG="$(grep -P '^v\d+\.\d+\.\d+$' tags | sort -rV | head -n1)"
+          echo "STABLE_TAG=${STABLE_TAG:-v0.0.0}" >> $GITHUB_ENV
+          LATEST_TAG="$(grep -P '^v\d+\.\d+\.\d+' tags | sort -rV | head -n1)"
+          echo "LATEST_TAG=${LATEST_TAG:-v0.0.0}" >> $GITHUB_ENV
+      - name: Update latest and stable tags
+        run: |
+          skopeo copy docker://${{ github.repository }}:${{ env.stable_tag }} docker://${{ github.repository }}:stable
+          skopeo copy docker://${{ github.repository }}:${{ env.latest_tag }} docker://${{ github.repository }}:latest

.github/workflows/development.yml

@@ -293,14 +293,18 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
       - name: Buildah build
         id: build-image
         uses: redhat-actions/buildah-build@v2
         with:
           image: ${{ github.event.repository.name }}
+          build-args: |
+            GUIDELLM_BUILD_TYPE=dev
           tags: "pr-${{ github.event.number }}"
           containerfiles: |
-            ./deploy/Containerfile
+            ./Containerfile
       - name: Push To ghcr.io
         id: push-to-ghcr
         uses: redhat-actions/push-to-registry@v2

.github/workflows/nightly.yml

@@ -251,14 +251,18 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
       - name: Buildah build
         id: build-image
         uses: redhat-actions/buildah-build@v2
         with:
           image: ${{ github.event.repository.name }}
+          build-args: |
+            GUIDELLM_BUILD_TYPE=nightly
           tags: nightly
           containerfiles: |
-            ./deploy/Containerfile
+            ./Containerfile
       - name: Push To ghcr.io
         id: push-to-ghcr
         uses: redhat-actions/push-to-registry@v2

.github/workflows/release-candidate.yml

@@ -295,15 +295,20 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - name: Get version from branch
+        run: echo "PACKAGE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV
       - name: Buildah build
         id: build-image
         uses: redhat-actions/buildah-build@v2
         with:
           image: ${{ github.event.repository.name }}
-          # TODO: Tag version
-          tags: latest
+          build-args: |
+            GUIDELLM_BUILD_TYPE=candidate
+          tags: ${{ env.package_version }}~rc
           containerfiles: |
-            ./deploy/Containerfile
+            ./Containerfile
       - name: Push To ghcr.io
         id: push-to-ghcr
         uses: redhat-actions/push-to-registry@v2

.github/workflows/release.yml

@@ -294,15 +294,20 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - name: Get version from branch
+        run: echo "PACKAGE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV
       - name: Buildah build
         id: build-image
         uses: redhat-actions/buildah-build@v2
         with:
           image: ${{ github.event.repository.name }}
-          # TODO: Tag version
-          tags: latest stable
+          build-args: |
+            GUIDELLM_BUILD_TYPE=release
+          tags: ${{ env.package_version }}
           containerfiles: |
-            ./deploy/Containerfile
+            ./Containerfile
       - name: Push To ghcr.io
         id: push-to-ghcr
         uses: redhat-actions/push-to-registry@v2

Containerfile

@@ -0,0 +1,65 @@
+# TODO: Update to official python-3.13-minimal image when available
+ARG BASE_IMAGE=quay.io/fedora/python-313-minimal:latest
+
+# release: take the last version and add a post if build iteration
+# candidate: increment to next minor, add 'rc' with build iteration
+# nightly: increment to next minor, add 'a' with build iteration
+# alpha: increment to next minor, add 'a' with build iteration
+# dev: increment to next minor, add 'dev' with build iteration
+ARG GUIDELLM_BUILD_TYPE=dev
+
+# Use a multi-stage build to create a lightweight production image
+FROM $BASE_IMAGE as builder
+
+# Switch to root for installing packages
+USER root
+
+# Install build tooling
+RUN dnf install -y git \
+    && /usr/bin/python3 -m venv /tmp/pdm \
+    && /tmp/pdm/bin/pip install --no-cache-dir -U pdm \
+    && ln -s /tmp/pdm/bin/pdm /usr/local/bin/pdm
+
+# Disable pdm update check
+# Set correct build type for versioning
+ENV PDM_CHECK_UPDATE=false \
+    GUIDELLM_BUILD_TYPE=$GUIDELLM_BUILD_TYPE
+
+# Copy repository files
+# Do this as late as possible to leverage layer caching
+COPY / /src
+
+# Install guidellm and locked dependencies
+RUN pdm use -p /src -f /opt/app-root \
+    && pdm install -p /src --check --prod --no-editable
+
+# Prod image
+FROM $BASE_IMAGE
+
+# Add guidellm bin to PATH
+# Argument defaults can be set with GUIDELLM_<ARG>
+ENV HOME="/home/guidellm" \
+    GUIDELLM_OUTPUT_PATH="/results/benchmarks.json"
+
+# Make sure root is the primary group
+USER 1001:0
+
+# Create the user home dir
+WORKDIR $HOME
+
+# Create a volume for results
+VOLUME /results
+
+# Metadata
+LABEL io.k8s.display-name="GuideLLM" \
+      org.opencontainers.image.description="GuideLLM Performance Benchmarking Container" \
+      org.opencontainers.image.source="https://github.com/vllm-project/guidellm" \
+      org.opencontainers.image.documentation="https://blog.vllm.ai/guidellm/stable" \
+      org.opencontainers.image.license="Apache-2.0"
+
+# Copy the virtual environment from the builder stage
+# Do this as late as possible to leverage layer caching
+COPY --chown=1001:0 --from=builder /opt/app-root /opt/app-root
+
+ENTRYPOINT [ "/opt/app-root/bin/guidellm" ]
+CMD [ "benchmark", "run" ]

README.md

@@ -71,6 +71,15 @@ podman run \
 
 Replace `latest` with `stable` for the newest tagged release or set a specific release if desired.
 
+#### Available Tags
+
+| Tags                                                                                       | Notes                                         |
+| ------------------------------------------------------------------------------------------ | --------------------------------------------- |
+| `nightly`                                                                                  | Built from `main` every night                 |
+| [`v0.3.0`](https://github.com/vllm-project/guidellm/releases/tag/v0.3.0) `stable` `latest` | -                                             |
+| [`v0.2.1`](https://github.com/vllm-project/guidellm/releases/tag/v0.2.1)                   | -                                             |
+| `pr-*`                                                                                     | Development builds (DO NOT USE IN PRODUCTION) |
+
 ### Quick Start
 
 #### 1. Start an OpenAI Compatible Server (vLLM)