describe testing in CONTRIBUTING.md

Author: rhysd

CONTRIBUTING.md

@@ -143,31 +143,34 @@ npm run serve
 git push
 ```
 
-## Maintain `popular_actions.go`
+## Maintain auto-generated sources
 
-[`popular_actions.go`](./popular_actions.go) is generated automatically with `go generate`. The command runs
-[`generate-popular-actions`](./scripts/generate-popular-actions) script.
+Some files are generated by scripts in [`scripts/`](./scripts) directory. These files are kept up-to-date by CI workflows.
 
-The script also can detect new releases of popular actions on GitHub by giving `-d` flag.
+### Maintain `popular_actions.go`
 
-Detecting new release and updating `popular_actions.go` are run weekly on CI by [`generate`](.github/workflows/generate.yaml)
-workflow. Runs can be checked [here](https://github.com/rhysd/actionlint/actions/workflows/generate.yaml).
+[`popular_actions.go`](./popular_actions.go) is a data set of metadata of popular actions hosted on GitHub. It is generated
+automatically with `go generate`. The command runs [`generate-popular-actions`](./scripts/generate-popular-actions) script.
 
-## Maintain `all_webhooks.go`
+The script also can detect new major releases of popular actions on GitHub by giving `-d` flag.
+
+The [`generate`](.github/workflows/generate.yaml) CI workflow weekly runs to detect new major releases and update
+`popular_actions.go`. Runs can be found [here](https://github.com/rhysd/actionlint/actions/workflows/generate.yaml).
+
+### Maintain `all_webhooks.go`
 
 [`all_webhooks.go`](./all_webhooks.go) is a table all webhooks supported by GitHub Actions to trigger workflows. Note that
 not all webhooks are supported by GitHub Actions.
 
-It is generated automatically with `go generate`. The command runs [`generate-webhook-events`](./scripts/generate-webhook-events)
-script.
+It is generated automatically with `go generate` running [`generate-webhook-events`](./scripts/generate-webhook-events) script.
 
 It fetches [`events-that-trigger-workflows.md`](https://github.com/github/docs/blob/main/content/actions/learn-github-actions/events-that-trigger-workflows.md),
 parses the markdown document, and extracts webhook names and their types. For more details, see
-[README.md at the script](./scripts/generate-webhook-events/README.md).
+[README.md at the script directory](./scripts/generate-webhook-events/README.md).
 
 Updating `all_webhooks.go` is run weekly on CI by [`generate`](.github/workflows/generate.yaml) workflow.
 
-## Maintain `actionlint-matcher.json`
+### Maintain `actionlint-matcher.json`
 
 [`actionlint-matcher.json`](.github/actionlint-matcher.json) is a matcher configuration to extract error annotations from outputs
 of `actionlint` command. See [the document](docs/usage.md#problem-matchers) for its usage.
@@ -178,7 +181,7 @@ file is not modified manually.
 It is generated by [`generate-actionlint-matcher`](./scripts/generate-actionlint-matcher) script. See the README.md file for the
 usage of the script and how to run the tests for it.
 
-## Maintain `availability.go`
+### Maintain `availability.go`
 
 [`availability.go`](./availability.go) is a table for conversion from workflow key (like `jobs.<job_id>.if`) to availability of
 contexts and special functions. GitHub Actions limits contexts and functions in certain places. For example:
@@ -194,8 +197,17 @@ Update for `availability.go` is run weekly on CI by [`generate`](.github/workflo
 
 ## Testing
 
-- All examples in ['Checks' document](docs/checks.md) are put in [the examples directory](testdata/examples) and tested in
-  [`linter_test.go`](linter_test.go).
-- I cloned GitHub top 1000 repositories and extracted 1400+ workflow files. And I tried actionlint with the collected workflow
-  files. All bugs found while the trial were fixed and I confirmed no more false positives.
-
+All tests are automated.
+
+- Unit tests are implemented in `*_test.go` files for testing the corresponding APIs. Test data for unit tests are put in
+  `testdata/` directory.
+- UI tests based on matching to error messages are implemented in `linter_test.go` and all test data are stored in `testdata/`
+  directory.
+  - `testdata/examples/` contains tests for all examples in ['Checks' document](docs/checks.md). `*.yaml` files are an input
+    workflow and `*.out` files are expected error messages.
+  - `testdata/ok/` contains 'OK' tests. All workflow files in this directory should cause no errors.
+  - `testdata/err/` contains 'Error' tests. Each `*.yaml` files are workflow inputs and corresponding `*.out` files are expected
+    error messages (one error per line).
+  - `testdata/projects/` contains 'Project' tests. Each directories represent a single project (meaning a repository on GitHub).
+    Corresponding `*.out` files are expected error messages. Empty `*.out` file means the test case should cause no errors.
+    'Project' test is used for use cases where multiple files are related (reusable workflows, local actions, config files, ...).