|
| 1 | +# Testing rules |
| 2 | + |
| 3 | +<!-- TODO(srawlins): Link to analyzer_testing, when published. --> |
| 4 | + |
| 5 | +The `analyzer_testing` package provides an API for testing analysis rules. Tests |
| 6 | +can be written concisely, encouraging the plugin author to write test cases with |
| 7 | +good coverage of possible Dart syntax, and the analysis rules themselves. |
| 8 | + |
| 9 | +## The test class |
| 10 | + |
| 11 | +Analysis rule tests that are written with the `analyzer_testing` package's |
| 12 | +support use a class hierarchy to specify shared variables, helper methods, and |
| 13 | +set-up and tear-down code. This is all based on the [`test_reflective_loader`][] |
| 14 | +package. Here is the basic structure: |
| 15 | + |
| 16 | + |
| 17 | +```dart |
| 18 | +import 'package:analyzer/src/lint/registry.dart'; |
| 19 | +import 'package:analyzer_testing/analysis_rule/analysis_rule.dart'; |
| 20 | +import 'package:my_rule/src/rules/my_rule.dart'; |
| 21 | +import 'package:test_reflective_loader/test_reflective_loader.dart'; |
| 22 | +
|
| 23 | +@reflectiveTest |
| 24 | +class MyRuleTest extends AnalysisRuleTest { |
| 25 | + @override |
| 26 | + void setUp() { |
| 27 | + Registry.ruleRegistry.registerLintRule(MyRule()); |
| 28 | + super.setUp(); |
| 29 | + } |
| 30 | +
|
| 31 | + @override |
| 32 | + String get analysisRule => 'my_rule'; |
| 33 | +
|
| 34 | + // Test cases go here. |
| 35 | +} |
| 36 | +``` |
| 37 | + |
| 38 | +This test file can be written anywhere in the `test` directory of the plugin |
| 39 | +package, maybe at `test/my_rule_test.dart`. |
| 40 | + |
| 41 | +In this code, we are testing the `my_rule` analysis rule built in |
| 42 | +[writing rules][], which reports any time an 'await expression' is found. |
| 43 | + |
| 44 | +This structure is different from the classic test structure used when writing |
| 45 | +tests with the `test` package, in which all tests are declared in anonymous |
| 46 | +closures passed to the `group` and `test` functions. Let's examine the |
| 47 | +components of the `MyRuleTest` class. |
| 48 | + |
| 49 | +* `class MyRuleTest extends AnalysisRuleTest` - The test class uses |
| 50 | + `AnalysisRuleTest`, from the `analyzer_testing` package, as a base. |
| 51 | + `AnalysisRuleTest` provides common functionality like `assertDiagnostics` and |
| 52 | + `newFile`. |
| 53 | +* `void setUp` - Override this method to provide some set-up code that is |
| 54 | + executed before each test. This method must call `super.setUp()`. This method |
| 55 | + is where we register the analysis rule that we are testing: |
| 56 | + `Registry.ruleRegistry.registerLintRule(MyRule());`. |
| 57 | +* `String get analysisRule` - This getter must be implemented, returning the |
| 58 | + analysis rule name, so that the test knows what analysis rule to expect. This |
| 59 | + is the name that the rule class passes up to the super-constructor. |
| 60 | + |
| 61 | +## The test cases |
| 62 | + |
| 63 | +The individual test cases are declared as instance methods of this class. Each |
| 64 | +method whose name starts with `test_` is registered as a test case. See the |
| 65 | +[`test_reflective_loader`][] package's documentation for more details. |
| 66 | + |
| 67 | +```dart |
| 68 | +@reflectiveTest |
| 69 | +class MyRuleTest extends AnalysisRuleTest { |
| 70 | + // ... |
| 71 | +
|
| 72 | + void test_has_await() async { |
| 73 | + await assertDiagnostics( |
| 74 | + r''' |
| 75 | +void f(Future<int> p) async { |
| 76 | + await p; |
| 77 | +} |
| 78 | +''', |
| 79 | + [lint(33, 5)], |
| 80 | + ); |
| 81 | + } |
| 82 | +
|
| 83 | + void test_no_await() async { |
| 84 | + await assertNoDiagnostics( |
| 85 | + r''' |
| 86 | +void f(Future<int> p) async { |
| 87 | + // No await. |
| 88 | +} |
| 89 | +'''); |
| 90 | + } |
| 91 | +} |
| 92 | +``` |
| 93 | + |
| 94 | +Let's look at the APIs used in these test cases: |
| 95 | + |
| 96 | +* `assertDiagnostics` - This is the primary assertion method used in analysis |
| 97 | + rule tests. It allows us to assert which diagnostics are reported, for some |
| 98 | + given Dart source code. The first argument is the source code, and the second |
| 99 | + is a list of expected diagnostics, `ExpectedDiagnostic` objects. Generally, |
| 100 | + `ExpectedDiagnostic` objects are not manually constructed. Instead, we use the |
| 101 | + `lint()` function: |
| 102 | +* `lint(33, 5)` - This utilitiy creates an expected diagnostic object |
| 103 | + representing the analysis rule specified by the `analysisRule` getter, which |
| 104 | + is expected at offset `33`, for a length of `5` characters. |
| 105 | +* `assertNoDiagnostics` - This is a convenience utility that asserts that _no_ |
| 106 | + diagnostcs are reported for the given source code. |
| 107 | + |
| 108 | +Most test cases can be written as simply as the two above, with a single call to |
| 109 | +`assertDiagnostics` or `assertNoDiagnostics`. |
| 110 | + |
| 111 | +Some test cases might involve code with compile-time errors, or warnings. (For |
| 112 | +example, you might want to verify that the analysis rule does not report when |
| 113 | +certain error conditions are present, so that the user can focus on fixing the |
| 114 | +error conditions, and not on spurious lint diagnostics.) Here is an example: |
| 115 | + |
| 116 | +```dart |
| 117 | + void test_has_await_in_non_async() async { |
| 118 | + await assertDiagnostics( |
| 119 | + r''' |
| 120 | +void f(Future<int> p) { |
| 121 | + await p; |
| 122 | +} |
| 123 | +''', |
| 124 | + [ |
| 125 | + // No lint is reported with this error. |
| 126 | + error(CompileTimeError.UNDEFINED_IDENTIFIER_AWAIT, 27, 5), |
| 127 | + ], |
| 128 | + ); |
| 129 | + } |
| 130 | +``` |
| 131 | + |
| 132 | +In this example, we assert that the only diagnostic reported for this code |
| 133 | +is an `CompileTimeError.UNDEFINED_IDENTIFIER_AWAIT` error. |
| 134 | + |
| 135 | +<!-- TODO(srawlins): In analyzer_testing: document writing multiple files with |
| 136 | + `newFile`, then link to it here. --> |
| 137 | +<!-- TODO(srawlins): In analyzer_testing: document writing a second package, |
| 138 | + then link to it here. --> |
| 139 | + |
| 140 | +## The entrypoint |
| 141 | + |
| 142 | +All of the test code above comes together when we register the test class in the |
| 143 | +test file's `main` function: |
| 144 | + |
| 145 | +```dart |
| 146 | +void main() { |
| 147 | + defineReflectiveSuite(() { |
| 148 | + defineReflectiveTests(MyRuleTest); |
| 149 | + }); |
| 150 | +} |
| 151 | +``` |
| 152 | + |
| 153 | +With this `main` function, tests can be run in the same way as class `test` |
| 154 | +package tests. They can be run in the usual ways, such as using the IDE, or by |
| 155 | +running `dart test` or `dart --enable-asserts test/my_rule_test.dart`. |
| 156 | + |
| 157 | +[writing rules]: https://github.com/dart-lang/sdk/blob/main/pkg/analysis_server_plugin/doc/writing_rules.md |
| 158 | +[`test_reflective_loader`]: https://pub.dev/packages/test_reflective_loader |
0 commit comments