Skip to content

Commit 819ec29

Browse files
DaltheCowDaltheCow
committed
Merge branch 'add-run-metadata-to-ui' of github.com:neuralmagic/guidellm into add-run-metadata-to-ui
2 parents be650fb + eae2355 commit 819ec29

File tree

14 files changed

+273
-53
lines changed

14 files changed

+273
-53
lines changed

‎.containerignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
.gitignore

‎.github/workflows/container-maintenance.yml‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
name: Container Image Maintenance
2+
3+
on:
4+
schedule:
5+
- cron: '0 2 * * 3' # Runs at 2am on Wednesdays
6+
workflow_dispatch: # Enables manual triggering of the workflow
7+
8+
# Only run one at a time
9+
concurrency:
10+
group: ${{ github.workflow }}
11+
12+
permissions:
13+
packages: write
14+
15+
jobs:
16+
cleanup-container-tags:
17+
runs-on: ubuntu-latest
18+
steps:
19+
- name: Delete PR and untagged images older than 2 weeks
20+
uses: snok/[email protected]
21+
with:
22+
account: ${{ github.repository_owner }}
23+
token: ${{ github.token }}
24+
image-names: ${{ github.event.repository.name }}
25+
image-tags: "pr-*"
26+
cut-off: 2w
27+
dry-run: false
28+
29+
push-container-tags:
30+
runs-on: ubuntu-latest
31+
needs: cleanup-container-tags
32+
if: always() # Run after cleanup even if it fails
33+
steps:
34+
- name: Log into ghcr.io
35+
uses: redhat-actions/podman-login@v1
36+
with:
37+
username: ${{ github.repository_owner }}
38+
password: ${{ github.token }}
39+
registry: ghcr.io/${{ github.repository_owner }}
40+
- name: Get list of tags
41+
run: |
42+
set -euo pipefail # Fail pipe if any command fails
43+
skopeo list-tags docker://ghcr.io/${{ github.repository }} | jq --raw-output '.Tags[]' > tags
44+
- name: Get latest release and rc tags
45+
run: |
46+
STABLE_TAG="$(grep -P '^v\d+\.\d+\.\d+$' tags | sort -rV | head -n1)"
47+
echo "stable_tag=${STABLE_TAG:-v0.0.0}" >> $GITHUB_ENV
48+
LATEST_TAG="$(grep -P '^v\d+\.\d+\.\d+' tags | sort -rV | head -n1)"
49+
echo "latest_tag=${LATEST_TAG:-v0.0.0}" >> $GITHUB_ENV
50+
- name: Update latest and stable tags
51+
run: |
52+
skopeo copy docker://ghcr.io/${{ github.repository }}:${{ env.stable_tag }} docker://ghcr.io/${{ github.repository }}:stable
53+
skopeo copy docker://ghcr.io/${{ github.repository }}:${{ env.latest_tag }} docker://ghcr.io/${{ github.repository }}:latest

‎.github/workflows/development.yml‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -293,14 +293,18 @@ jobs:
293293
steps:
294294
- name: Checkout
295295
uses: actions/checkout@v4
296+
with:
297+
fetch-depth: 0
296298
- name: Buildah build
297299
id: build-image
298300
uses: redhat-actions/buildah-build@v2
299301
with:
300302
image: ${{ github.event.repository.name }}
303+
build-args: |
304+
GUIDELLM_BUILD_TYPE=dev
301305
tags: "pr-${{ github.event.number }}"
302306
containerfiles: |
303-
./deploy/Containerfile
307+
./Containerfile
304308
- name: Push To ghcr.io
305309
id: push-to-ghcr
306310
uses: redhat-actions/push-to-registry@v2

‎.github/workflows/nightly.yml‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -251,14 +251,18 @@ jobs:
251251
steps:
252252
- name: Checkout
253253
uses: actions/checkout@v4
254+
with:
255+
fetch-depth: 0
254256
- name: Buildah build
255257
id: build-image
256258
uses: redhat-actions/buildah-build@v2
257259
with:
258260
image: ${{ github.event.repository.name }}
261+
build-args: |
262+
GUIDELLM_BUILD_TYPE=nightly
259263
tags: nightly
260264
containerfiles: |
261-
./deploy/Containerfile
265+
./Containerfile
262266
- name: Push To ghcr.io
263267
id: push-to-ghcr
264268
uses: redhat-actions/push-to-registry@v2

‎.github/workflows/release-candidate.yml‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -295,15 +295,20 @@ jobs:
295295
steps:
296296
- name: Checkout
297297
uses: actions/checkout@v4
298+
with:
299+
fetch-depth: 0
300+
- name: Get version from branch
301+
run: echo "PACKAGE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV
298302
- name: Buildah build
299303
id: build-image
300304
uses: redhat-actions/buildah-build@v2
301305
with:
302306
image: ${{ github.event.repository.name }}
303-
# TODO: Tag version
304-
tags: latest
307+
build-args: |
308+
GUIDELLM_BUILD_TYPE=candidate
309+
tags: ${{ env.package_version }}~rc
305310
containerfiles: |
306-
./deploy/Containerfile
311+
./Containerfile
307312
- name: Push To ghcr.io
308313
id: push-to-ghcr
309314
uses: redhat-actions/push-to-registry@v2

‎.github/workflows/release.yml‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -294,15 +294,20 @@ jobs:
294294
steps:
295295
- name: Checkout
296296
uses: actions/checkout@v4
297+
with:
298+
fetch-depth: 0
299+
- name: Get version from branch
300+
run: echo "PACKAGE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV
297301
- name: Buildah build
298302
id: build-image
299303
uses: redhat-actions/buildah-build@v2
300304
with:
301305
image: ${{ github.event.repository.name }}
302-
# TODO: Tag version
303-
tags: latest stable
306+
build-args: |
307+
GUIDELLM_BUILD_TYPE=release
308+
tags: ${{ env.package_version }}
304309
containerfiles: |
305-
./deploy/Containerfile
310+
./Containerfile
306311
- name: Push To ghcr.io
307312
id: push-to-ghcr
308313
uses: redhat-actions/push-to-registry@v2

‎Containerfile‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# TODO: Update to official python-3.13-minimal image when available
2+
ARG BASE_IMAGE=quay.io/fedora/python-313-minimal:latest
3+
4+
# release: take the last version and add a post if build iteration
5+
# candidate: increment to next minor, add 'rc' with build iteration
6+
# nightly: increment to next minor, add 'a' with build iteration
7+
# alpha: increment to next minor, add 'a' with build iteration
8+
# dev: increment to next minor, add 'dev' with build iteration
9+
ARG GUIDELLM_BUILD_TYPE=dev
10+
11+
# Use a multi-stage build to create a lightweight production image
12+
FROM $BASE_IMAGE as builder
13+
14+
# Switch to root for installing packages
15+
USER root
16+
17+
# Install build tooling
18+
RUN dnf install -y git \
19+
&& /usr/bin/python3 -m venv /tmp/pdm \
20+
&& /tmp/pdm/bin/pip install --no-cache-dir -U pdm \
21+
&& ln -s /tmp/pdm/bin/pdm /usr/local/bin/pdm
22+
23+
# Disable pdm update check
24+
# Set correct build type for versioning
25+
ENV PDM_CHECK_UPDATE=false \
26+
GUIDELLM_BUILD_TYPE=$GUIDELLM_BUILD_TYPE
27+
28+
# Copy repository files
29+
# Do this as late as possible to leverage layer caching
30+
COPY / /src
31+
32+
# Install guidellm and locked dependencies
33+
RUN pdm use -p /src -f /opt/app-root \
34+
&& pdm install -p /src --check --prod --no-editable
35+
36+
# Prod image
37+
FROM $BASE_IMAGE
38+
39+
# Add guidellm bin to PATH
40+
# Argument defaults can be set with GUIDELLM_<ARG>
41+
ENV HOME="/home/guidellm" \
42+
GUIDELLM_OUTPUT_PATH="/results/benchmarks.json"
43+
44+
# Make sure root is the primary group
45+
USER 1001:0
46+
47+
# Create the user home dir
48+
WORKDIR $HOME
49+
50+
# Create a volume for results
51+
VOLUME /results
52+
53+
# Metadata
54+
LABEL io.k8s.display-name="GuideLLM" \
55+
org.opencontainers.image.description="GuideLLM Performance Benchmarking Container" \
56+
org.opencontainers.image.source="https://github.com/vllm-project/guidellm" \
57+
org.opencontainers.image.documentation="https://blog.vllm.ai/guidellm/stable" \
58+
org.opencontainers.image.license="Apache-2.0"
59+
60+
# Copy the virtual environment from the builder stage
61+
# Do this as late as possible to leverage layer caching
62+
COPY --chown=1001:0 --from=builder /opt/app-root /opt/app-root
63+
64+
ENTRYPOINT [ "/opt/app-root/bin/guidellm" ]
65+
CMD [ "benchmark", "run" ]

‎README.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,15 @@ podman run \
7171
7272
Replace `latest` with `stable` for the newest tagged release or set a specific release if desired.
7373

74+
#### Available Tags
75+
76+
| Tags | Notes |
77+
| ------------------------------------------------------------------------------------------ | --------------------------------------------- |
78+
| `nightly` | Built from `main` every night |
79+
| [`v0.3.0`](https://github.com/vllm-project/guidellm/releases/tag/v0.3.0) `stable` `latest` | - |
80+
| [`v0.2.1`](https://github.com/vllm-project/guidellm/releases/tag/v0.2.1) | - |
81+
| `pr-*` | Development builds (DO NOT USE IN PRODUCTION) |
82+
7483
### Quick Start
7584

7685
#### 1. Start an OpenAI Compatible Server (vLLM)

‎deploy/Containerfile‎

Lines changed: 0 additions & 42 deletions
This file was deleted.

‎docs/assets/sample-output1.png‎

324 KB
Loading

0 commit comments

Comments
 (0)