diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index cb09705a70..b63c3f817f 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -8,6 +8,7 @@ Note: Before submitting a pull request, please open an issue for discussion if y - [ ] I have followed guidelines from [CONTRIBUTING.MD](https://github.com/GoogleCloudPlatform/nodejs-docs-samples/blob/main/CONTRIBUTING.md) and [Samples Style Guide](https://googlecloudplatform.github.io/samples-style-guide/) - [ ] **Tests** pass: `npm test` (see [Testing](https://github.com/GoogleCloudPlatform/nodejs-docs-samples/blob/main/CONTRIBUTING.md#run-the-tests-for-a-single-sample)) - [ ] **Lint** pass: `npm run lint` (see [Style](https://github.com/GoogleCloudPlatform/nodejs-docs-samples/blob/main/CONTRIBUTING.md#style)) +- [ ] **Required CI tests** pass (see [CI testing](https://github.com/GoogleCloudPlatform/nodejs-docs-samples/blob/main/CONTRIBUTING.md#ci-testing)) - [ ] These samples need a new **API enabled** in testing projects to pass (let us know which ones) - [ ] These samples need a new/updated **env vars** in testing projects set to pass (let us know which ones) - [ ] This pull request is from a branch created directly off of `GoogleCloudPlatform/nodejs-docs-samples`. Not a fork. @@ -15,3 +16,5 @@ Note: Before submitting a pull request, please open an issue for discussion if y - [ ] This sample adds a new sample directory, and I created [GitHub Actions workflow](https://github.com/GoogleCloudPlatform/nodejs-docs-samples/blob/main/CONTRIBUTING.md#adding-new-samples) for this sample - [ ] This sample adds a new **Product API**, and I updated the [Blunderbuss issue/PR auto-assigner](https://github.com/GoogleCloudPlatform/nodejs-docs-samples/blob/main/.github/blunderbuss.yml) with the codeowners for this sample - [ ] Please **merge** this PR for me once it is approved + +> **Note**: Any check with `(dev)`, `(experimental)`, or `(legacy)` can be ignored and should **not block** your PR from merging (see [CI testing](https://github.com/GoogleCloudPlatform/nodejs-docs-samples/blob/main/CONTRIBUTING.md#ci-testing)). diff --git a/.github/workflows/custard-ci-dev.yaml b/.github/workflows/custard-ci-dev.yaml index e2550d0c51..47d08508d9 100644 --- a/.github/workflows/custard-ci-dev.yaml +++ b/.github/workflows/custard-ci-dev.yaml @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -name: Custard CI (dev) +name: (legacy dev) Custard CI on: push: branches: @@ -27,7 +27,7 @@ env: jobs: affected: - name: Finding affected tests + name: (legacy) Finding affected tests runs-on: ubuntu-latest timeout-minutes: 2 outputs: @@ -61,7 +61,8 @@ jobs: echo "setups=$(./cloud-samples-tools/bin/custard setup-files .github/config/nodejs-dev.jsonc paths.txt)" >> $GITHUB_OUTPUT nodejs-test: - name: test + if: github.event.pull_request && github.event.pull_request.head.repo.fork == false + name: (legacy) test needs: affected runs-on: ubuntu-latest timeout-minutes: 120 # 2 hours hard limit diff --git a/.github/workflows/custard-ci.yaml b/.github/workflows/custard-ci.yaml index de7ca11875..a63e570c33 100644 --- a/.github/workflows/custard-ci.yaml +++ b/.github/workflows/custard-ci.yaml @@ -12,6 +12,9 @@ # See the License for the specific language governing permissions and # limitations under the License. +# IMPORTANT: DO NOT CHANGE THE NAME OF THIS WORKFLOW! +# The workflow named "Custard CI" triggers both custard-run.yaml and custard-run-dev.yaml. +# TODO: To sunset old tests remove `affected`, `lint`, and `test` jobs (only keep `region-tags`). name: Custard CI on: push: @@ -28,7 +31,7 @@ env: # TODO: remove all jobs except region-tags (should be tested by custard-run workflows) jobs: affected: - name: Finding affected tests + name: (legacy) Finding affected tests runs-on: ubuntu-latest timeout-minutes: 2 outputs: @@ -119,6 +122,8 @@ jobs: - run: ./.github/workflows/utils/region-tags-tests.sh test: + if: github.event.pull_request && github.event.pull_request.head.repo.fork == false + name: (legacy) test needs: affected runs-on: ubuntu-latest timeout-minutes: 120 # 2 hours hard limit diff --git a/.github/workflows/custard-run-dev.yaml b/.github/workflows/custard-run-dev.yaml index 978496c3f2..a7f9957ae9 100644 --- a/.github/workflows/custard-run-dev.yaml +++ b/.github/workflows/custard-run-dev.yaml @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -name: (experimental / dev) Custard run +name: (dev) Custard run on: # Run tests when a pull request is created or updated. @@ -52,7 +52,7 @@ jobs: head-sha: ${{ github.event.workflow_run.head_sha || inputs.ref || github.sha }} config-file: .github/config/nodejs-dev.jsonc paths: ${{ (inputs.run-all && '.') || inputs.paths || '' }} - check-name: (experimental / dev) Custard CI + check-name: (dev) Custard CI / tests create-check-if: ${{ !!github.event.workflow_run }} test: @@ -77,7 +77,7 @@ jobs: id: queued with: sha: ${{ github.event.workflow_run.head_sha || inputs.ref || github.sha }} - name: (experimental / dev) Custard CI / ${{ github.job }} (${{ matrix.path }}) + name: (dev) Custard CI / ${{ github.job }} (${{ matrix.path }}) job-name: ${{ github.job }} (${{ matrix.path }}) if: ${{ !!github.event.workflow_run }} - name: Checkout diff --git a/.github/workflows/custard-run.yaml b/.github/workflows/custard-run.yaml index 3edafec81e..c5fc5c94d2 100644 --- a/.github/workflows/custard-run.yaml +++ b/.github/workflows/custard-run.yaml @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -name: (experimental) Custard run +name: Custard run on: # Run tests when a pull request is created or updated. @@ -57,7 +57,7 @@ jobs: head-sha: ${{ github.event.workflow_run.head_sha || inputs.ref || github.sha }} config-file: .github/config/nodejs.jsonc paths: ${{ (inputs.run-all && '.') || inputs.paths || '' }} - check-name: (experimental) Custard CI + check-name: Custard CI / tests create-check-if: ${{ !!github.event.workflow_run }} lint: @@ -73,7 +73,7 @@ jobs: with: sha: ${{ github.event.workflow_run.head_sha || inputs.ref || github.sha }} status: in_progress - name: (experimental) Custard CI / ${{ github.job }} + name: Custard CI / ${{ github.job }} if: ${{ !!github.event.workflow_run }} - name: Checkout uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4 @@ -123,7 +123,7 @@ jobs: id: queued with: sha: ${{ github.event.workflow_run.head_sha || inputs.ref || github.sha }} - name: (experimental) Custard CI / ${{ github.job }} (${{ matrix.path }}) + name: Custard CI / ${{ github.job }} (${{ matrix.path }}) job-name: ${{ github.job }} (${{ matrix.path }}) if: ${{ !!github.event.workflow_run }} - name: Checkout diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index af25dd5ff7..cfdb25e9da 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -64,34 +64,35 @@ const assert = require('node:assert/strict'); ### CI testing -For new samples, a GitHub Actions workflow should be created to run your tests -on the CI system: - -1. Check that your new samples and sample tests are on a branch created directly from this repo `GoogleCloudPlatform/nodejs-docs-samples`. Not a fork. - -1. Add an entry to - [.github/workflows/utils/workflows.json](https://github.com/GoogleCloudPlatform/nodejs-docs-samples/blob/main/.github/workflows/utils/workflows.json) - matching the directory with your sample code. - -1. From the root of the repo, generate a new workflow in the - [workflows](https://github.com/GoogleCloudPlatform/nodejs-docs-samples/blob/main/.github/workflows) - directory. You can specify a `path` to only generate the specific workflow, - e.g. `cloud-tasks`. If the path is omitted, all workflows will be generated. - - node .github/workflows/utils/generate.js -s [path] - -> **Note** There are some existing samples that use an alternative CI system. It -> is recommended to use GitHub Actions for new samples, but these instructions -> are provided below for your reference. -> -> Add a **build configuration file (`.cfg`)** for your samples in -> **[`.kokoro/`](https://github.com/GoogleCloudPlatform/nodejs-docs-samples/tree/main/.kokoro)**. -> Check existing config files for the right format. -> -> All tests need a corresponding job file outside of GitHub. If you are a -> Googler, please provide the CL alongside your PR. See the internal codelab for -> Kokoro for details. If you don't work at Google, the person reviewing your PR -> will create the job config for you. +> **tl;dr**: Any check with `(dev)`, `(experimental)`, or `(legacy)` can be ignored and should **not block** your PR from merging. + +This repository uses the 🍮 **Custard CI** from +[`GoogleCloudPlatform/cloud-samples-tools`](https://github.com/GoogleCloudPlatform/cloud-samples-tools) +to set up and run tests. + +First, it checks which files changed in a PR to find which packages were affected. +In this repo, a _package_ is defined as a directory containing a `package.json` file. + +If a global file (not belonging to a package) is changed, all packages are marked as affected. +Global files are can contain repo-level configurations that could affect other packages. +Typically, contributors should only modify files in a package. + +Only affected packages are linted and tested. +For tests, we use a +[matrix strategy](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/running-variations-of-jobs-in-a-workflow) +to generate isolated jobs for each affected test. + +We're working to improve the testing infrastructure, while keeping tests stable. +This means you'll sometimes see some new experimental tests, you can generally ignore them. +Or two versions of one test running while we make sure the new version works well. + +Here's a list of which tests are required and which ones you can ignore. +* `Custard CI / lint`: (**required**) runs only for affected packages +* `Custard CI / tests`: (**required**) this runs until all tests have finished +* `Custard CI / test (package)`: (**required**) runs only for affected packages +* `(dev) Custard CI / test (package)`: (_ignore_) this test has errors we're working on +* `(experimental) Custard CI / ...`: (_ignore_) this is a new unstable version +* `Custard CI / (legacy) ...`: (_ignore_) this is the old version running while we make sure the new one works ### TypeScript Support