Improve documentation about extended tests (#14320)

* Improve documentation about extended tests

* Improvements

* clarify extent of overage

* Update docs/source/contributor-guide/testing.md

---------

Co-authored-by: Oleks V <[email protected]>

Author: alamb

.github/workflows/extended.yml

@@ -23,9 +23,11 @@ concurrency:
 
 # https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#running-your-pull_request-workflow-when-a-pull-request-merges
 #
-# this job is intended to only run only on the main branch as it is time consuming
-# and should not fail often. However, it is important coverage to ensure correctness
-# in the (very rare) event of a hash failure or sqlite query failure.
+# These jobs only run on the `main` branch as they are time consuming
+# and should not fail often.
+#
+# We still run them as they provide important coverage to ensure correctness
+# in the (very rare) event of a hash failure or sqlite library query failure.
 on:
   # Run on all commits to main
   push:

docs/source/contributor-guide/testing.md

@@ -58,6 +58,18 @@ Like similar systems such as [DuckDB](https://duckdb.org/dev/testing), DataFusio
 
 DataFusion has integrated [sqlite's test suite](https://sqlite.org/sqllogictest/doc/trunk/about.wiki) as a supplemental test suite that is run whenever a PR is merged into DataFusion. To run it manually please refer to the [README](https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/README.md#running-tests-sqlite) file for instructions.
 
+## Extended Tests
+
+In addition to the standard CI test suite that is run on all PRs prior to merge,
+DataFusion has "extended" tests (defined in [extended.yml]) that are run on each
+commit to `main`. These tests rarely fail but take significantly longer to run
+than the standard test suite and add important test coverage such as that the
+code works when there are hash collisions as well as running the relevant
+portions of the entire [sqlite test suite].
+
+[extended.yml]: https://github.com/apache/datafusion/blob/main/.github/workflows/extended.yml
+[sqlite test suite]: https://www.sqlite.org/sqllogictest/dir?ci=tip
+
 ## Rust Integration Tests
 
 There are several tests of the public interface of the DataFusion library in the [tests](https://github.com/apache/datafusion/tree/main/datafusion/core/tests) directory.