Skip to content

Commit e50b939

Browse files
alambJefffrey
andauthored
Docs: Enhance testing documentation with examples and links (#18851)
## Which issue does this PR close? - part of #7013 ## Rationale for this change I was trying to find the right documentation to point people (and AI tools) at for running the tests and found some improvements to the existing testing page that would be nice: https://datafusion.apache.org/contributor-guide/testing.html ## What changes are included in this PR? Added examples and links for running tests and using test utilities in the contributor guide. ## Are these changes tested? I downloaded it locally ## Are there any user-facing changes? Better testing guide --------- Co-authored-by: Jeffrey Vo <[email protected]>
1 parent d3ae1a7 commit e50b939

File tree

1 file changed

+27
-5
lines changed

1 file changed

+27
-5
lines changed

docs/source/contributor-guide/testing.md

Lines changed: 27 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -46,19 +46,41 @@ cargo nextest run
4646
## Unit tests
4747

4848
Tests for code in an individual module are defined in the same source file with a `test` module, following Rust convention.
49-
The [test_util](https://github.com/apache/datafusion/tree/main/datafusion/common/src/test_util.rs) module provides useful macros to write unit tests effectively, such as `assert_batches_sorted_eq` and `assert_batches_eq` for RecordBatches and `assert_contains` / `assert_not_contains` which are used extensively in the codebase.
49+
50+
For example, to run tests in the `datafusion` crate:
51+
52+
```shell
53+
cargo test -p datafusion
54+
```
55+
56+
The [test_util] module provides useful macros to write unit tests effectively, such as [`assert_batches_sorted_eq`] and [`assert_batches_eq`] for RecordBatches and [`assert_contains`] / [`assert_not_contains`] which are used extensively in the codebase.
57+
58+
[test_util]: https://github.com/apache/datafusion/tree/main/datafusion/common/src/test_util.rs
59+
[`assert_batches_sorted_eq`]: https://docs.rs/datafusion/latest/datafusion/macro.assert_batches_sorted_eq.html
60+
[`assert_batches_eq`]: https://docs.rs/datafusion/latest/datafusion/macro.assert_batches_eq.html
61+
[`assert_contains`]: https://docs.rs/datafusion/latest/datafusion/common/macro.assert_contains.html
62+
[`assert_not_contains`]: https://docs.rs/datafusion/latest/datafusion/common/macro.assert_not_contains.html
5063

5164
## sqllogictests Tests
5265

53-
DataFusion's SQL implementation is tested using [sqllogictest](https://github.com/apache/datafusion/tree/main/datafusion/sqllogictest) which are run like other tests using `cargo test --test sqllogictests`.
66+
DataFusion's SQL implementation is tested using [sqllogictest](https://github.com/apache/datafusion/tree/main/datafusion/sqllogictest). You can run these tests with commands like:
67+
68+
```shell
69+
# Run all tests
70+
cargo test --profile=ci --test sqllogictests
71+
# Run a specific test file
72+
cargo test --profile=ci --test sqllogictests -- aggregate.slt
73+
# Run and update expected outputs
74+
cargo test --profile=ci --test sqllogictests -- --complete
75+
```
5476

55-
`sqllogictests` tests may be less convenient for new contributors who are familiar with writing `.rs` tests as they require learning another tool. However, `sqllogictest` based tests are much easier to develop and maintain as they 1) do not require a slow recompile/link cycle and 2) can be automatically updated via `cargo test --test sqllogictests -- --complete`.
77+
`sqllogictests` may be less convenient for new contributors who are familiar with writing `.rs` tests as they require learning another tool. However, `sqllogictest` based tests are much easier to develop and maintain as they 1) do not require a slow recompile/link cycle and 2) can be automatically updated.
5678

5779
Like similar systems such as [DuckDB](https://duckdb.org/dev/testing), DataFusion has chosen to trade off a slightly higher barrier to contribution for longer term maintainability.
5880

5981
DataFusion has integrated [sqlite's test suite](https://sqlite.org/sqllogictest/doc/trunk/about.wiki) as a supplemental test suite that is run whenever a PR is merged into DataFusion. To run it manually please refer to the [README](https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/README.md#running-tests-sqlite) file for instructions.
6082

61-
## Snapshot testing
83+
## Snapshot testing (`cargo insta`)
6284

6385
[Insta](https://github.com/mitsuhiko/insta) is used for snapshot testing. Snapshots are generated
6486
and compared on each test run. If the output changes, tests will fail.
@@ -90,7 +112,7 @@ There are several public interface tests for the DataFusion library in the [test
90112
You can run these tests individually using `cargo` as normal command such as
91113

92114
```shell
93-
cargo test -p datafusion --test parquet_exec
115+
cargo test -p datafusion --test parquet_integration
94116
```
95117

96118
## SQL "Fuzz" testing

0 commit comments

Comments
 (0)