Skip to content

Merge Dev #745

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
anandhu-eng merged 9 commits into from
Dec 27, 2025
Merged

Merge Dev #745

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
03593cb
Support custom port for remote_run_cmds (#740)
amd-arsuresh Dec 25, 2025
2c46828
Update test-mlperf-inference-rgat.yml
arjunsuresh Dec 26, 2025
02720ef
Improvements to mlc remote_run (#741)
arjunsuresh Dec 26, 2025
9292943
[Automated Commit] Document script/remote-run-commands/meta.yaml [sk…
github-actions[bot] Dec 26, 2025
dd9258d
Cleanups for mlcrr (#742)
arjunsuresh Dec 27, 2025
2539d98
Update test-nvidia-mlperf-inference-implementations.yml (#743)
arjunsuresh Dec 27, 2025
70f8d3d
Merge branch 'main' into dev
arjunsuresh Dec 27, 2025
a70dd3f
Ignore md5sha1sum version detect failure on macos (#748)
arjunsuresh Dec 27, 2025
7cfe24d
Refactor script search logic in script automation (#747)
amd-arsuresh Dec 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/test-mlperf-inference-mlcommons-cpp-resnet50.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,9 @@ on:
- '!**.md'

jobs:
build:
run-mlperf-inference-resnet50-cpp:
name: MLPerf inference MLCommons C++ ResNet50
if: github.repository_owner == 'mlcommons'
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-mlperf-inference-rgat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest]
python-version: [ "3.14" ]
python-version: [ "3.12" ]
backend: [ "pytorch" ]
implementation: [ "python" ]

Expand Down
17 changes: 6 additions & 11 deletions .github/workflows/test-nvidia-mlperf-inference-implementations.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,93 +4,88 @@
pull_request_target:
branches: [ "main", "dev" ]
paths:
- '.github/workflows/test-mlperf-inference-resnet50.yml'
- '**'
- '.github/workflows/test-nvidia-mlperf-inference-implementations.yml'
- 'script/app-mlperf-inference-nvidia/**'
- '!**.md'

jobs:
run_nvidia:
if: github.repository_owner == 'gateoverflow'
timeout-minutes: 1440
if: github.repository_owner == 'mlcommons'
runs-on:
- self-hosted
- linux
- x64
- cuda
- ${{ matrix.system }}
- ubuntu-latest
strategy:
fail-fast: false
matrix:
system: [ "GO-i9" ]
# system: [ "mlc-server" ]
python-version: [ "3.12" ]
model: [ "resnet50" ]
exclude:
- model: gptj-99.9
- system: phoenix1
- system: GO-i91

steps:
- name: Set up SSH
run: |
mkdir -p ~/.ssh
chmod 700 ~/.ssh

echo "${{ secrets.SSH_PRIVATE_KEY }}" > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519

ssh-keyscan -H ${{ secrets.SSH_HOST }} >> ~/.ssh/known_hosts
#ssh-keyscan -p ${{ secrets.SSH_PORT }} -H ${{ secrets.SSH_HOST }} >> ~/.ssh/known_hosts
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}
- name: Install mlcflow
run: |
pip install mlcflow
pip install tabulate
- name: Pull MLOps repo
shell: bash
env:
REPO: ${{ github.event.pull_request.head.repo.html_url }}
BRANCH: ${{ github.event.pull_request.head.ref }}
run: |
mlc pull repo "$REPO" --branch="$BRANCH"

- name: Test MLPerf Inference NVIDIA ${{ matrix.model }}
env:
gpu_name: rtx_4090
run: |
# Set hw_name based on matrix.system
if [ "${{ matrix.system }}" = "GO-spr" ]; then
hw_name="RTX4090x2"
gpu_name=rtx_4090
docker_string=" --docker --docker_recreate=yes"
elif [ "${{ matrix.system }}" = "mlc-server" ]; then
hw_name="H100x8"
gpu_name=h100
docker_string=" "
else
hw_name="RTX4090x1"
gpu_name=rtx_4090
docker_string=" --docker"
fi
if [ "${{ matrix.model }}" = "bert-99.9" ]; then
submission_preprocessor_args=" --noinfer-low-accuracy-results"
else
submission_preprocessor_args=""
fi
category="datacenter,edge"
if [ -f "gh_action/bin/deactivate" ]; then source gh_action/bin/deactivate; fi
python3 -m venv gh_action
source gh_action/bin/activate
export MLC_REPOS=$HOME/GH_MLC
pip install --upgrade mlcflow
mlc pull repo mlcommons@mlperf-automations --branch=dev

mlcrr run-mlperf,inference,_all-scenarios,_submission,_full,_r5.1-dev --remote_host=${{ secrets.SSH_HOST }} --remote_user=${{ secrets.SSH_USER }} --remote_port=${{ secrets.SSH_PORT }} --preprocess_submission=yes --pull_changes=yes --pull_inference_changes=yes --execution_mode=valid --gpu_name=$gpu_name --pull_changes=yes --pull_inference_changes=yes --model=${{ matrix.model }} --submitter="GATEOverflow" --hw_name=$hw_name --implementation=nvidia --backend=tensorrt --category=$category --division=closed --docker_dt --docker_mlc_repo=mlcommons@mlperf-automations --docker_mlc_repo_branch=dev --adr.compiler.tags=gcc --device=cuda --use_model_from_host=yes --use_dataset_from_host=yes --results_dir=$HOME/gh_action_results --submission_dir=$HOME/gh_action_submissions --clean $docker_string $submission_preprocessor_args --quiet
mlcrr run-mlperf,inference,_all-scenarios,_submission,_full,_r5.1-dev --docker --remote_host=${{ secrets.SSH_HOST }} --remote_user=${{ secrets.SSH_USER }} --remote_port=${{ secrets.SSH_PORT }} --remote_ssh_key_file=$HOME/.ssh/id_ed25519 --remote_skip_host_verify --preprocess_submission=yes --pull_changes=yes --pull_inference_changes=yes --execution_mode=valid --gpu_name=$gpu_name --pull_changes=yes --pull_inference_changes=yes --model=${{ matrix.model }} --submitter="GATEOverflow" --hw_name=$hw_name --implementation=nvidia --backend=tensorrt --category=$category --division=closed --docker_dt --docker_mlc_repo=mlcommons@mlperf-automations --docker_mlc_repo_branch=dev --adr.compiler.tags=gcc --device=cuda --use_model_from_host=yes --use_dataset_from_host=yes --results_dir=$HOME/gh_action_results --submission_dir=$HOME/gh_action_submissions --clean $docker_string $submission_preprocessor_args --quiet
#mlcr push,github,mlperf,inference,submission --repo_url=https://github.com/mlcommons/mlperf_inference_unofficial_submissions_v5.1 --repo_branch=auto-update --commit_message="Results from GH action on NVIDIA_$hw_name" --quiet --submission_dir=$HOME/gh_action_submissions --hw_name=$hw_name
#mlcr push,github,mlperf,inference,submission --repo_url=https://github.com/GATEOverflow/mlperf_inference_submissions_v5.0 --repo_branch=main --commit_message="Results from GH actions on NVIDIA_$hw_name" --quiet --submission_dir=$HOME/gh_action_submissions --hw_name=$hw_name

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium test

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

2 changes: 1 addition & 1 deletion automation/script/docker.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -388,7 +388,7 @@ def docker_run(self_module, i):

# Ensure Docker is available
r = self_module.action_object.access(
{'action': 'run', 'automation': 'script', 'tags': "get,docker"})
{'action': 'run', 'automation': 'script', 'tags': "get,docker", 'quiet': True})
if r['return'] > 0:
return r

Expand Down
Loading
Loading