add a workflow for running UMF benchmarks #1060
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| name: Compute Benchmarks | ||
|
Contributor
There was a problem hiding this comment. teeny, tiny thing to consider: for UMF we don't execute "Compute Benchmarks" - we execute our benchmarks written with gtest. We could consider renaming it (in all places, incl. writing comments, etc...) |
||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| pr_no: | ||
| description: PR number (if 0, it'll run on the main) | ||
| type: number | ||
| bench_script_params: | ||
| description: Parameters passed to script executing benchmark | ||
| type: string | ||
| required: false | ||
| default: '' | ||
| upload_report: | ||
| description: 'Upload HTML report' | ||
| type: boolean | ||
| required: false | ||
| default: false | ||
|
|
||
| permissions: | ||
| contents: read | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| manual: | ||
| name: Compute Benchmarks | ||
| uses: ./.github/workflows/reusable_benchmarks.yml | ||
| with: | ||
| pr_no: ${{ inputs.pr_no }} | ||
| bench_script_params: ${{ inputs.bench_script_params }} | ||
| upload_report: ${{ inputs.upload_report }} | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,80 +1,163 @@ | ||
| # Executes benchmarks implemented in this repository | ||
lukaszstolarczuk marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| # using scripts for benchmark results visualisation, | ||
| # which are downloaded from Unified Runtime repository. | ||
| name: Benchmarks | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| pr_no: | ||
| # even though this is a number, this is a workaround for issues with | ||
| # reusable workflow calls that result in "Unexpected value '0'" error. | ||
| type: string | ||
| default: '0' | ||
lukaszstolarczuk marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| bench_script_params: | ||
| required: false | ||
| type: string | ||
| default: '' | ||
| upload_report: | ||
| required: false | ||
| type: boolean | ||
| default: false | ||
|
|
||
| permissions: | ||
| contents: read | ||
| pull-requests: write | ||
|
|
||
| env: | ||
| UMF_DIR: "${{github.workspace}}/umf-repo" | ||
| BUILD_DIR : "${{github.workspace}}/umf-repo/build" | ||
|
|
||
| jobs: | ||
| benchmarks: | ||
| name: Benchmarks | ||
| strategy: | ||
| matrix: | ||
| os: ['ubuntu-latest'] | ||
| runs-on: ${{matrix.os}} | ||
|
|
||
| steps: | ||
| # Workspace on self-hosted runners is not cleaned automatically. | ||
| # We have to delete the files created outside of using actions. | ||
| - name: Cleanup self-hosted workspace | ||
| if: always() | ||
| run: | | ||
| ls -la ./ | ||
| rm -rf ./* || true | ||
|
|
||
| - name: Add comment to PR | ||
| uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1 | ||
| if: ${{ always() && inputs.pr_no != 0 }} | ||
PatKamin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| with: | ||
| script: | | ||
| const pr_no = '${{ inputs.pr_no }}'; | ||
| const url = '${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}'; | ||
| const params = '${{ inputs.bench_script_params }}'; | ||
| const body = `Compute Benchmarks run (with params: ${params}):\n${url}`; | ||
|
|
||
| github.rest.issues.createComment({ | ||
| issue_number: pr_no, | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| body: body | ||
| }) | ||
|
|
||
| - name: Checkout UMF | ||
| uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 | ||
| with: | ||
| path: ${{env.UMF_DIR}} | ||
| fetch-depth: 0 | ||
|
|
||
| # We need to fetch special ref for proper PR's merge commit. Note, this ref may be absent if the PR is already merged. | ||
| - name: Fetch PR's merge commit | ||
| if: ${{ inputs.pr_no != 0 }} | ||
| working-directory: ${{env.UMF_DIR}} | ||
| env: | ||
| PR_NO: ${{ inputs.pr_no }} | ||
| run: | | ||
| git fetch -- https://github.com/${{github.repository}} +refs/pull/${PR_NO}/*:refs/remotes/origin/pr/${PR_NO}/* | ||
| git checkout origin/pr/${PR_NO}/merge | ||
| git rev-parse origin/pr/${PR_NO}/merge | ||
|
|
||
| - name: Install apt packages | ||
| run: | | ||
| sudo apt-get update | ||
| sudo apt-get install -y cmake libhwloc-dev libnuma-dev libtbb-dev | ||
|
|
||
| - name: Configure build | ||
| run: > | ||
| cmake | ||
| -S ${{env.UMF_DIR}} | ||
| -B ${{env.BUILD_DIR}} | ||
| -DCMAKE_BUILD_TYPE=Release | ||
| -DUMF_BUILD_SHARED_LIBRARY=ON | ||
| -DUMF_BUILD_BENCHMARKS=ON | ||
| -DUMF_BUILD_BENCHMARKS_MT=ON | ||
| -DUMF_BUILD_TESTS=OFF | ||
lukaszstolarczuk marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| -DUMF_FORMAT_CODE_STYLE=OFF | ||
| -DUMF_DEVELOPER_MODE=OFF | ||
| -DUMF_BUILD_LEVEL_ZERO_PROVIDER=ON | ||
| -DUMF_BUILD_CUDA_PROVIDER=ON | ||
| -DUMF_BUILD_LIBUMF_POOL_DISJOINT=ON | ||
| -DUMF_BUILD_LIBUMF_POOL_JEMALLOC=ON | ||
| -DUMF_BUILD_EXAMPLES=OFF | ||
|
|
||
| - name: Build UMF | ||
| run: cmake --build ${{env.BUILD_DIR}} -j $(nproc) | ||
|
|
||
| # We are going to clone Unified Runtime repository in order to run | ||
| # the most up-to-date UR scripts for benchmark data visualisation | ||
| - name: Checkout UR | ||
lukaszstolarczuk marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 | ||
| with: | ||
| repository: oneapi-src/unified-runtime | ||
| path: ur-repo | ||
| fetch-depth: 1 | ||
| fetch-tags: false | ||
|
|
||
| - name: Install pip packages for benchmarking scripts from UR | ||
| run: | | ||
| pip install --force-reinstall -r ${{github.workspace}}/ur-repo/third_party/benchmark_requirements.txt | ||
|
|
||
| - name: Run dedicated for UMF benchmarking scripts from UR | ||
| id: benchmarks | ||
| working-directory: ${{env.BUILD_DIR}} | ||
| run: > | ||
| ${{ github.workspace }}/ur-repo/scripts/benchmarks/main.py | ||
| ~/bench_workdir | ||
| --umf ${{env.BUILD_DIR}} | ||
| ${{ inputs.upload_report && '--output-html' || '' }} | ||
| ${{ inputs.bench_script_params }} | ||
|
|
||
| - name: Add comment to PR | ||
| uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1 | ||
| if: ${{ always() && inputs.pr_no != 0 }} | ||
| with: | ||
| script: | | ||
| let markdown = "" | ||
| try { | ||
| const fs = require('fs'); | ||
| markdown = fs.readFileSync('${{env.BUILD_DIR}}/benchmark_results.md', 'utf8'); | ||
| } catch(err) { | ||
| } | ||
|
|
||
| const pr_no = '${{ inputs.pr_no }}'; | ||
| const url = '${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}'; | ||
| const test_status = '${{ steps.benchmarks.outcome }}'; | ||
| const job_status = '${{ job.status }}'; | ||
| const params = '${{ inputs.bench_script_params }}'; | ||
| const body = `Compute Benchmarks run (${params}):\n${url}\nJob status: ${job_status}. Test status: ${test_status}.\n ${markdown}`; | ||
|
|
||
| github.rest.issues.createComment({ | ||
| issue_number: pr_no, | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| body: body | ||
| }) | ||
|
|
||
| - name: Upload HTML report | ||
| if: ${{ always() && inputs.upload_report }} | ||
| uses: actions/cache/save@1bd1e32a3bdc45362d1e726936510720a7c30a57 # v4.2.0 | ||
| with: | ||
| path: ${{env.BUILD_DIR}}/benchmark_results.html | ||
| key: benchmark-results-${{ github.run_id }} | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.