Skip to content

Commit 6bc885a

Browse files
authored
Document minimum requirements for test engines (#3118)
This commit adds a subsection to the User Guide documenting the requirements for test engines for interoperability with IDEs and build tools.
1 parent 17b9180 commit 6bc885a

File tree

1 file changed

+42
-0
lines changed
  • documentation/src/docs/asciidoc/user-guide/advanced-topics

1 file changed

+42
-0
lines changed

documentation/src/docs/asciidoc/user-guide/advanced-topics/engines.adoc

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,3 +78,45 @@ For example, the `junit-jupiter-engine` module registers its
7878
`org.junit.jupiter.engine.JupiterTestEngine` in a file named
7979
`org.junit.platform.engine.TestEngine` within the `/META-INF/services` folder in the
8080
`junit-jupiter-engine` JAR.
81+
82+
[[test-engines-requirements]]
83+
==== Requirements
84+
85+
NOTE: The words "must", "must not", "required", "shall", "shall not", "should", "should
86+
not", "recommended", "may", and "optional" in this section are to be interpreted as
87+
described in https://www.ietf.org/rfc/rfc2119.txt[RFC 2119.]
88+
89+
[[test-engines-requirements-mandatory]]
90+
===== Mandatory requirements
91+
92+
For interoperability with build tools and IDEs, `TestEngine` implementations must adhere
93+
to the following requirements:
94+
95+
* The `TestDescriptor` returned from `TestEngine.discover()` _must_ be the root of a tree
96+
of `TestDescriptor` instances. This implies that there _must not_ be any cycles between
97+
a node and its descendants.
98+
* A `TestEngine` _must_ be able to discover `UniqueIdSelectors` for any unique ID that it
99+
previously generated and returned from `TestEngine.discover()`. This enables selecting a
100+
subset of tests to execute or rerun.
101+
* The `executionSkipped`, `executionStarted`, and `executionFinished` methods of the
102+
`EngineExecutionListener` passed to `TestEngine.execute()` _must_ be called for every
103+
`TestDescriptor` node in the tree returned from `TestEngine.discover()` at most
104+
once. Parent nodes _must_ be reported as started before their children and as finished
105+
after their children. If a node is reported as skipped, there _must not_ be any events
106+
reported for its descendants.
107+
108+
[[test-engines-requirements-enhanced-compatibility]]
109+
===== Enhanced compatibility
110+
111+
Adhering to the following requirements is optional but recommended for enhanced
112+
compatibility with build tools and IDEs:
113+
114+
* Unless to indicate an empty discovery result, the `TestDescriptor` returned from
115+
`TestEngine.discover()` _should_ have children rather than being completely dynamic.
116+
This allows tools to display the structure of the tests and to select a subset of tests
117+
to execute.
118+
* When resolving `UniqueIdSelectors`, a `TestEngine` _should_ only return `TestDescriptor`
119+
instances with matching unique IDs including their ancestors but _may_ return additional
120+
siblings or other nodes that are required for the execution of the selected tests.
121+
* `TestEngines` _should_ support <<running-tests-tags, tagging>> tests and containers so
122+
that tag filters can be applied when discovering tests.

0 commit comments

Comments
 (0)