[CI][UR] Bring up UR benchmarking results dashboard in intel.github.io #17475
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| name: Run Benchmarks | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: '0 1 * * *' # 2 hrs earlier than sycl-nightly.yml | ||
| workflow_call: | ||
| inputs: | ||
| commit_hash: | ||
| type: string | ||
| required: false | ||
| upload_results: | ||
| type: string # true/false: workflow_dispatch does not support booleans | ||
| required: true | ||
| runner: | ||
| type: string | ||
| required: true | ||
| backend: | ||
| type: string | ||
| required: true | ||
| reset_intel_gpu: | ||
| type: string # true/false: workflow_dispatch does not support booleans | ||
| required: true | ||
| default: true | ||
|
|
||
| workflow_dispatch: | ||
| inputs: | ||
| commit_hash: | ||
| description: Commit hash to build intel/llvm from | ||
| type: string | ||
| required: false | ||
| default: '' | ||
| upload_results: | ||
| description: 'Save and upload results' | ||
| type: choice | ||
| options: | ||
| - false | ||
| - true | ||
| default: true | ||
| runner: | ||
| type: choice | ||
| options: | ||
| - '["PVC_PERF"]' | ||
| backend: | ||
| description: Backend to use | ||
| type: choice | ||
| options: | ||
| - 'level_zero:gpu' | ||
| # TODO L0 V2 support | ||
| reset_intel_gpu: | ||
| description: Reset Intel GPUs | ||
| type: choice | ||
| options: | ||
| - false | ||
| - true | ||
| default: true | ||
|
|
||
| permissions: read-all | ||
|
|
||
| jobs: | ||
| build_sycl: | ||
| name: Build SYCL from PR | ||
| if: inputs.commit_hash != '' | ||
|
There was a problem hiding this comment. techincally passing empty // FYI, trigger type can be used to establish if nightly or not There was a problem hiding this comment. Good catch! think I resolved this via the latest commit: I've set default values on commit_hash to always be a string // trigger type can be used to determine if a job was scheduled, but I've chosen to use commit_hash here incase the user themselves wants to use the nightly build of sycl manually There was a problem hiding this comment. hmm tbh, I'm not sure if setting There was a problem hiding this comment. maybe I'm missing something, but can we easily read the hash commit from PR? I know we used There was a problem hiding this comment. The sycl-linux-build.yml job in intel/llvm doesn't support compiling commits from other branches, thus we cannot access commits from PR branches as of now: I have chosen to not stir the nest here, but this will probably be a future change that involes changing sycl-linux-build.yml There was a problem hiding this comment. hmm... so what should I pass here to check my PR? or is it intended as a post-merge check? There was a problem hiding this comment. Admittedly I dont think that'd be possible right now 😅 We will figure something out though, given that this PR is no longer urgent, I'll make a PR to change sycl-linux-build.yml so that we can actually accomplish this |
||
| uses: ./.github/workflows/sycl-linux-build.yml | ||
| with: | ||
| build_ref: ${{ inputs.commit_hash }} | ||
| build_cache_root: "/__w/" | ||
| build_artifact_suffix: "default" | ||
| build_cache_suffix: "default" | ||
| # Docker image has last nightly pre-installed and added to the PATH | ||
| build_image: "ghcr.io/intel/llvm/sycl_ubuntu2404_nightly:latest" | ||
| cc: clang | ||
| cxx: clang++ | ||
| changes: '[]' | ||
|
|
||
| run_benchmarks_build: | ||
| name: Run Benchmarks (on PR Build) | ||
| needs: [ build_sycl ] | ||
| if: inputs.commit_hash != '' | ||
| strategy: | ||
| matrix: | ||
| # Set default values if not specified: | ||
| include: | ||
| - runner: ${{ inputs.runner || '["PVC_PERF"]' }} | ||
| backend: ${{ inputs.backend || 'level_zero:gpu' }} | ||
| reset_intel_gpu: ${{ inputs.reset_intel_gpu || 'true' }} | ||
| ref: ${{ inputs.commit_hash }} | ||
| uses: ./.github/workflows/sycl-linux-run-tests.yml | ||
| secrets: inherit | ||
| with: | ||
| # TODO support other benchmarks | ||
| name: Run compute-benchmarks (${{ matrix.runner }}, ${{ matrix.backend }}) | ||
| runner: ${{ matrix.runner }} | ||
| image: ghcr.io/intel/llvm/sycl_ubuntu2404_nightly:latest | ||
| image_options: -u 1001 --device=/dev/dri -v /dev/dri/by-path:/dev/dri/by-path --privileged --cap-add SYS_ADMIN | ||
| target_devices: ${{ matrix.backend }} | ||
| reset_intel_gpu: ${{ matrix.reset_intel_gpu }} | ||
| tests_selector: benchmark_v2 | ||
| benchmark_upload_results: ${{ inputs.upload_results }} | ||
| benchmark_build_hash: ${{ inputs.commit_hash }} | ||
| repo_ref: ${{ matrix.ref }} | ||
| devops_ref: ${{ github.ref }} | ||
| sycl_toolchain_artifact: sycl_linux_default | ||
| sycl_toolchain_archive: ${{ needs.build_sycl.outputs.artifact_archive_name }} | ||
| sycl_toolchain_decompress_command: ${{ needs.build_sycl.outputs.artifact_decompress_command }} | ||
|
|
||
| run_benchmarks_nightly: | ||
| name: Run Benchmarks (on Nightly Build) | ||
| if: inputs.commit_hash == '' | ||
| strategy: | ||
| matrix: | ||
| # Set default values if not specified: | ||
| include: | ||
| - runner: ${{ inputs.runner || '["PVC_PERF"]' }} | ||
| backend: ${{ inputs.backend || 'level_zero:gpu' }} | ||
| reset_intel_gpu: ${{ inputs.reset_intel_gpu || 'true' }} | ||
| uses: ./.github/workflows/sycl-linux-run-tests.yml | ||
| secrets: inherit | ||
| with: | ||
| # TODO support other benchmarks | ||
| name: Run compute-benchmarks (${{ matrix.runner }}, ${{ matrix.backend }}) | ||
| runner: ${{ matrix.runner }} | ||
| image: ghcr.io/intel/llvm/sycl_ubuntu2404_nightly:latest | ||
| image_options: -u 1001 --device=/dev/dri -v /dev/dri/by-path:/dev/dri/by-path --privileged --cap-add SYS_ADMIN | ||
| target_devices: ${{ matrix.backend }} | ||
| reset_intel_gpu: ${{ matrix.reset_intel_gpu }} | ||
| tests_selector: benchmark_v2 | ||
| benchmark_upload_results: ${{ inputs.upload_results }} | ||
| repo_ref: ${{ github.ref }} | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,52 +1,47 @@ | ||
| name: Test benchmark.yml using sycl-benchmark-aggregate | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| commit_hash: | ||
| description: Specific commit hash to build SYCL from | ||
| type: string | ||
| required: false | ||
| upload_results: | ||
| description: 'Save and upload results' | ||
| type: choice | ||
| options: | ||
| - false | ||
| - true | ||
| default: true | ||
| runner: | ||
| type: choice | ||
| options: | ||
| - '["PVC_PERF"]' | ||
| backend: | ||
| description: Backend to use | ||
| type: choice | ||
| options: | ||
| - 'level_zero:gpu' | ||
| # TODO L0 V2 support | ||
| reset_intel_gpu: | ||
| description: Reset Intel GPUs | ||
| type: choice | ||
| options: | ||
| - false | ||
| - true | ||
| default: true | ||
|
|
||
| permissions: read-all | ||
|
|
||
| jobs: | ||
| aggregate: | ||
| name: Test benchmark.yml | ||
| uses: ./.github/workflows/benchmark.yml | ||
| secrets: inherit | ||
| with: | ||
| commit_hash: ${{ inputs.commit_hash }} | ||
| upload_results: ${{ inputs.upload_results }} | ||
| runner: ${{ inputs.runner }} | ||
| backend: ${{ inputs.backend }} | ||
| reset_intel_gpu: ${{ inputs.reset_intel_gpu }} | ||
kswiecicki marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.