[ci] ARM Taichi #8654

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

johnnynunez wants to merge 26 commits into taichi-dev:master from johnnynunez:arm

.github/workflows/build.yaml

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -28,7 +28,7 @@ on:
  
            default: false

            type: boolean

          python:

            description: 'JSON encoded python versions need building wheel. e.g.: ["3.7"]'

            description: 'JSON encoded python versions need building wheel. e.g.: ["3.12"]'

            required: true

            type: string

    @@ -83,7 +83,7 @@ jobs:
  
          - name: Workaround checkout Needed single revision issue

            run: git submodule foreach 'git rev-parse HEAD > /dev/null 2>&1 || rm -rf $PWD' || true

          - uses: actions/checkout@v3

          - uses: actions/checkout@v4

            with:

              fetch-depth: '0'

              submodules: 'recursive'

    @@ -92,7 +92,7 @@ jobs:
  
            run: ./build.py ${NIGHTLY:+--nightly} --python=${{ matrix.python }} ${SHOULD_TAG_CONFIG:+--tag-config}

          - name: Upload Built Wheel

            uses: shallwefootball/s3-upload-action@v1.3.3

            uses: shallwefootball/s3-upload-action@master

            with:

              aws_key_id: ${{ secrets.BOT_MINIO_ACCESS_KEY }}

              aws_secret_access_key: ${{ secrets.BOT_MINIO_SECRET_KEY }}

    @@ -131,7 +131,7 @@ jobs:
  
          - name: Workaround checkout Needed single revision issue

            run: git submodule foreach 'git rev-parse HEAD > /dev/null 2>&1 || rm -rf $PWD' || true

          - uses: actions/checkout@v3

          - uses: actions/checkout@v4

            with:

              submodules: 'recursive'

              fetch-depth: '0'

    @@ -147,7 +147,7 @@ jobs:
  
              SHOULD_TAG_CONFIG: ${{ !matrix.designated && 'yes' || '' }}

          - name: Upload Built Wheel

            uses: shallwefootball/s3-upload-action@v1.3.3

            uses: shallwefootball/s3-upload-action@master

            with:

              aws_key_id: ${{ secrets.BOT_MINIO_ACCESS_KEY }}

              aws_secret_access_key: ${{ secrets.BOT_MINIO_SECRET_KEY }}

    @@ -177,7 +177,7 @@ jobs:
  
          - name: Workaround checkout Needed single revision issue

            run: git submodule foreach 'git rev-parse HEAD > /dev/null 2>&1 || rm -rf $PWD' || true

          - uses: actions/checkout@v3

          - uses: actions/checkout@v4

            with:

              submodules: 'recursive'

              fetch-depth: '0'

    @@ -192,7 +192,7 @@ jobs:
  
                /home/dev/taichi/build.py ${NIGHTLY:+--nightly} --python=${{ matrix.python }} --tag-local=amd

          - name: Upload Built Wheel

            uses: shallwefootball/s3-upload-action@v1.3.3

            uses: shallwefootball/s3-upload-action@master

            with:

              aws_key_id: ${{ secrets.BOT_MINIO_ACCESS_KEY }}

              aws_secret_access_key: ${{ secrets.BOT_MINIO_SECRET_KEY }}

    @@ -240,15 +240,15 @@ jobs:
  
              git submodule foreach --recursive 'git rev-parse HEAD || rm -rf $PWD'

              $LASTEXITCODE = 0

          - uses: actions/checkout@v3

          - uses: actions/checkout@v4

            with:

              fetch-depth: '0'

              submodules: 'recursive'

          - uses: actions/setup-python@v4

          - uses: actions/setup-python@v5

            with:

              # force a 3.7 is ok, build.py will handle actual python env

              python-version: 3.7

              # force a 3.12 is ok, build.py will handle actual python env

              python-version: 3.12

          - name: Build

            shell: pwsh

    @@ -262,7 +262,7 @@ jobs:
  
              SHOULD_TAG_CONFIG: ${{ !matrix.designated && 'yes' || '' }}

          - name: Upload Built Wheel

            uses: shallwefootball/s3-upload-action@v1.3.3

            uses: shallwefootball/s3-upload-action@master

            with:

              aws_key_id: ${{ secrets.BOT_MINIO_ACCESS_KEY }}

              aws_secret_access_key: ${{ secrets.BOT_MINIO_SECRET_KEY }}

    @@ -289,6 +289,7 @@ jobs:
  
            _designated: ['']

            exclude:

            - python: "3.7"

            - python: "3.8"

            include:

            - _designated: ''

              designated: designated

    @@ -309,7 +310,7 @@ jobs:
  
          - name: Workaround checkout Needed single revision issue

            run: git submodule foreach 'git rev-parse HEAD > /dev/null 2>&1 || rm -rf $PWD' || true

          - uses: actions/checkout@v3

          - uses: actions/checkout@v4

            with:

              fetch-depth: '0'

              submodules: 'recursive'

    @@ -320,7 +321,7 @@ jobs:
  
              ./build.py ${NIGHTLY:+--nightly} --python=${{ matrix.python }}

          - name: Upload Built Wheel

            uses: shallwefootball/s3-upload-action@v1.3.3

            uses: shallwefootball/s3-upload-action@master

            with:

              aws_key_id: ${{ secrets.BOT_MINIO_ACCESS_KEY }}

              aws_secret_access_key: ${{ secrets.BOT_MINIO_SECRET_KEY }}

    @@ -337,7 +338,7 @@ jobs:
  
          - name: Workaround checkout Needed single revision issue

            run: git submodule foreach 'git rev-parse HEAD > /dev/null 2>&1 || rm -rf $PWD' || true

          - uses: actions/checkout@v3

          - uses: actions/checkout@v4

            with:

              fetch-depth: '0'

              submodules: 'recursive'

.github/workflows/ghstack_land.yml

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -17,13 +17,13 @@ jobs:
  
            EOF

        - name: Show Github Event Path Json

          run: 'cat $GITHUB_EVENT_PATH || true'

        - uses: actions/checkout@v3

        - uses: actions/checkout@v4

          with:

            token: ${{ secrets.GARDENER_PAT }}

            fetch-depth: '0'

        - uses: actions/setup-python@v4

        - uses: actions/setup-python@v5

          with:

            python-version: 3.9

            python-version: 3.12

        - name: Check Current CI Status

          run: |

            pip install requests ghstack

.github/workflows/initiator.yaml

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -58,7 +58,7 @@ jobs:
  
        with:

          build_id: ${{ needs.preparation.outputs.build_id }}

          nightly: false

          python: '["3.9", "3.10", "3.11"]'

          python: '["3.10", "3.11", "3.12", "3.13"]'

        secrets:

          BOT_MINIO_ACCESS_KEY: ${{ secrets.BOT_MINIO_ACCESS_KEY }}

          BOT_MINIO_SECRET_KEY: ${{ secrets.BOT_MINIO_SECRET_KEY }}

.github/workflows/issue_comment.yml

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -8,7 +8,7 @@ jobs:
  
        runs-on: ubuntu-latest

        steps:

        - name: Slash Command Dispatch - Triage

          uses: peter-evans/slash-command-dispatch@v3

          uses: peter-evans/slash-command-dispatch@v4

          with:

            token: ${{ secrets.GARDENER_PAT }}

            reaction-token: ${{ secrets.GARDENER_PAT }}

    @@ -21,7 +21,7 @@ jobs:
  
              rerun-failed

        - name: Slash Command Dispatch - Repo - Triage

          uses: peter-evans/slash-command-dispatch@v3

          uses: peter-evans/slash-command-dispatch@v4

          with:

            token: ${{ secrets.GARDENER_PAT }}

            reaction-token: ${{ secrets.GARDENER_PAT }}

    @@ -31,7 +31,7 @@ jobs:
  
              benchmark

        - name: Slash Command Dispatch - Repo - Write

          uses: peter-evans/slash-command-dispatch@v3

          uses: peter-evans/slash-command-dispatch@v4

          with:

            token: ${{ secrets.GARDENER_PAT }}

            reaction-token: ${{ secrets.GARDENER_PAT }}

.github/workflows/perf.yml

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -23,7 +23,7 @@ jobs:
  
          - name: Workaround checkout Needed single revision issue

            run: git submodule foreach 'git rev-parse HEAD > /dev/null 2>&1 || rm -rf $PWD' || true

          - uses: actions/checkout@v2

          - uses: actions/checkout@v4

            with:

              submodules: "recursive"

              fetch-depth: '0'

    @@ -43,7 +43,7 @@ jobs:
  
                /home/dev/taichi/.github/workflows/scripts/build.py

            env:

              PY: '3.8'

              PY: '3.12'

              PROJECT_NAME: taichi

              TAICHI_CMAKE_ARGS: >-

                -DTI_WITH_OPENGL:BOOL=ON

    @@ -82,7 +82,7 @@ jobs:
  
                /home/dev/taichi/.github/workflows/scripts/unix-perf-mon.sh

            env:

              PY: '3.8'

              PY: '3.12'

              BENCHMARK_UPLOAD_TOKEN: ${{ secrets.BENCHMARK_UPLOAD_TOKEN }}

              GITHUB_EVENT_ACTION: ${{ github.event.action }}

              GITHUB_TOKEN: ${{ secrets.GARDENER_PAT }}

.github/workflows/pull_request.yml

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -9,10 +9,10 @@ jobs:
  
        if: ${{ github.event.pull_request }}

        runs-on: ubuntu-latest

        steps:

          - uses: actions/checkout@v2

          - uses: actions/setup-python@v4

          - uses: actions/checkout@v4

          - uses: actions/setup-python@v5

            with:

              python-version: 3.8

              python-version: 3.12

          - name: Install Dependencies

            run: pip install semver GitPython PyGithub

.github/workflows/release.yml

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -34,7 +34,7 @@ jobs:
  
        if: github.repository_owner == 'taichi-dev'

        runs-on: ubuntu-latest

        steps:

          - uses: actions/checkout@v3

          - uses: actions/checkout@v4

          - name: Save new version

            run: |

    @@ -57,16 +57,15 @@ jobs:
  
            run: |

              if [ -n "$RELEASE_VERSION" ]; then

                # For production release, we run on five python versions.

                echo 'matrix={"include":[{"name":"taichi","python":"3.7"},{"name":"taichi","python":"3.8"},{"name":"taichi","python":"3.9"},{"name":"taichi","python":"3.10"},{"name":"taichi","python":"3.11"},{"name":"taichi","python":"3.12"}]}"' >> $GITHUB_OUTPUT

                echo 'matrix={"include":[{"name":"taichi","python":"3.9"}, {"name":"taichi","python":"3.10"},{"name":"taichi","python":"3.11"},{"name":"taichi","python":"3.12"},{"name":"taichi","python":"3.13"}]}"' >> $GITHUB_OUTPUT

                # M1 only supports 3.8, 3.9, and 3.10(conda), so change matrix.

                echo 'matrix_osx={"include":[{"name":"taichi","python":"3.8"},{"name":"taichi","python":"3.9"},{"name":"taichi","python":"3.10"},{"name":"taichi","python":"3.11"},{"name":"taichi","python":"3.12"}]}"' >> $GITHUB_OUTPUT

                echo 'matrix_osx={"include":[{"name":"taichi","python":"3.9"}, {"name":"taichi","python":"3.10"},{"name":"taichi","python":"3.11"},{"name":"taichi","python":"3.12"}, {"name":"taichi","python":"3.13"}]}"' >> $GITHUB_OUTPUT

              else

                # For nightly release, we run on three python versions.

                echo 'matrix={"include":[{"name":"taichi-nightly","python":"3.7"},{"name":"taichi-nightly","python":"3.8"},{"name":"taichi-nightly","python":"3.9"},{"name":"taichi-nightly","python":"3.10"},{"name":"taichi-nightly","python":"3.11"},{"name":"taichi","python":"3.12"}]}"' >> $GITHUB_OUTPUT

                echo 'matrix={"include":[{"name":"taichi-nightly","python":"3.10"},{"name":"taichi-nightly","python":"3.11"},{"name":"taichi","python":"3.12"},{"name":"taichi","python":"3.13"}]}"' >> $GITHUB_OUTPUT

                # M1 only supports 3.8 and 3.10(conda), so change matrix.

                echo 'matrix_osx={"include":[{"name":"taichi-nightly","python":"3.8"},{"name":"taichi-nightly","python":"3.9"},{"name":"taichi-nightly","python":"3.10"},{"name":"taichi-nightly","python":"3.11"},{"name":"taichi","python":"3.12"}]}"' >> $GITHUB_OUTPUT

                echo 'matrix_osx={"include":[{"name":"taichi-nightly","python":"3.10"},{"name":"taichi-nightly","python":"3.11"},{"name":"taichi","python":"3.12"},{"name":"taichi","python":"3.13"}]}"' >> $GITHUB_OUTPUT

              fi

      build_and_test_linux:

    @@ -85,7 +84,65 @@ jobs:
  
            -DTI_BUILD_TESTS:BOOL=ON

        steps:

          - uses: actions/checkout@v3

          - uses: actions/checkout@v4

            with:

              submodules: 'recursive'

              fetch-depth: '0'

          - name: Prepare Environment

            run: |

              . .github/workflows/scripts/common-utils.sh

              prepare-build-cache

              echo CI_DOCKER_RUN_EXTRA_ARGS="-v $(pwd):/home/dev/taichi" >> $GITHUB_ENV

          - name: Build

            id: build

            run: |

              . .github/workflows/scripts/common-utils.sh

              ci-docker-run-gpu --name taichi-build \

                registry.botmaster.tgr/taichi-build-cuda:${{ env.CI_IMAGE_VERSION }} \

                /home/dev/taichi/.github/workflows/scripts/build.py $NIGHTLY

              # Get wheel filename

              WHEEL_NAME=$(ls dist/*.whl | head -n 1 | xargs basename)

              echo "wheel_name=$WHEEL_NAME" >> $GITHUB_OUTPUT

              echo "Found wheel: $WHEEL_NAME"

          - name: Archive Wheel Artifacts

            uses: actions/upload-artifact@v4

            with:

              name: ${{ steps.build.outputs.wheel_name }}

              path: dist/${{ steps.build.outputs.wheel_name }}

              retention-days: 20

          - name: Test

            run: |

              . .github/workflows/scripts/common-utils.sh

              ci-docker-run-gpu --name taichi-test \

                 registry.botmaster.tgr/taichi-test-cuda:${{ env.CI_IMAGE_VERSION }} \

                 /home/dev/taichi/.github/workflows/scripts/unix_test.sh

            env:

              PY: ${{ matrix.python }}

      build_and_test_linux_arm:

        name: Build and Upload (linux only)

        needs: matrix_prep

        strategy:

          fail-fast: false

          matrix: ${{ fromJson(needs.matrix_prep.outputs.matrix) }}

        runs-on: [ ubuntu-24.04-arm, vulkan, release ]

        env:

          PY: ${{ matrix.python }}

          PROJECT_NAME: ${{ matrix.name }}

          TAICHI_CMAKE_ARGS: >-

            -DTI_WITH_OPENGL:BOOL=ON

            -DTI_WITH_VULKAN:BOOL=ON

            -DTI_BUILD_TESTS:BOOL=ON

        steps:

          - uses: actions/checkout@v4

            with:

              submodules: 'recursive'

              fetch-depth: '0'

    @@ -142,7 +199,7 @@ jobs:
  
      #       -DTI_WITH_OPENGL:BOOL=OFF

      #       -DTI_BUILD_TESTS:BOOL=ON

      #   steps:

      #     - uses: actions/checkout@v3

      #     - uses: actions/checkout@v4

      #       with:

      #         submodules: 'recursive'

      #         fetch-depth: '0'

    @@ -194,7 +251,7 @@ jobs:
  
            -DTI_BUILD_TESTS:BOOL=ON

        steps:

          - uses: actions/checkout@v3

          - uses: actions/checkout@v4

            with:

              submodules: 'recursive'

              fetch-depth: '0'

    @@ -255,12 +312,12 @@ jobs:
  
              git config --system core.longpaths true

              git submodule foreach 'git rev-parse HEAD > /dev/null 2>&1 || rm -rf $PWD'

          - uses: actions/checkout@v3

          - uses: actions/checkout@v4

            with:

              submodules: 'recursive'

              fetch-depth: '0'

          - uses: actions/setup-python@v4

          - uses: actions/setup-python@v5

            with:

              python-version: ${{ matrix.python }}

    @@ -307,13 +364,13 @@ jobs:
  
      upload_to_pypi:

        name: Upload release to PyPI

        needs: [build_and_test_linux, build_and_test_m1, build_and_test_windows]

        needs: [build_and_test_linux, build_and_test_m1, build_and_test_windows, build_and_test_linux_arm]

        runs-on: ubuntu-latest

        steps:

          - uses: actions/checkout@v3

          - uses: actions/setup-python@v4

          - uses: actions/checkout@v4

          - uses: actions/setup-python@v5

            with:

              python-version: 3.9

              python-version: 3.12

          - name: Download all artifacts

            uses: actions/download-artifact@v4

    @@ -350,13 +407,13 @@ jobs:
  
        runs-on: ubuntu-latest

        if: ${{ github.event.inputs.version }}

        steps:

          - uses: actions/checkout@v3

          - uses: actions/checkout@v4

            with:

              fetch-depth: 0

          - uses: actions/setup-python@v4

          - uses: actions/setup-python@v5

            with:

              python-version: 3.9

              python-version: 3.12

          - name: Generate Changelog

            id: changelog

.github/workflows/scripts/ti_build/dep.py

-Original file line number
+Diff line change
@@ Expand Up @@
         elif name.endswith(".tar.gz") or name.endswith(".tgz"):
             outdir.mkdir(parents=True, exist_ok=True)
             tar("-xzf", local_cached, "-C", outdir, f"--strip-components={strip}")
+        elif name.endswith(".tar.xz"):
+            outdir.mkdir(parents=True, exist_ok=True)
+            tar("-xJf", local_cached, "-C", outdir, f"--strip-components={strip}")
         elif name.endswith(".sh"):
             bash(local_cached, *args)
         elif "." not in name and args is not None:
@@ Expand Down @@

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ci] ARM Taichi #8654

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

Uh oh!