Skip to content

Commit 75fc7df

Browse files
committed
Add test add-on command resolution docs (#3616)
### Motivation Step towards #3421 This PR adds the second part of the test add-on documentation, which is command resolution. I tried to explain in a generic way, showing some examples of what command resolution is supposed to do. Please let me know if this is not clear enough and if the examples need to be enhanced.
1 parent e5244f6 commit 75fc7df

File tree

1 file changed

+112
-1
lines changed

1 file changed

+112
-1
lines changed

jekyll/test_framework_addons.markdown

Lines changed: 112 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ test frameworks, like [Active Support test case](rails-add-on) and [RSpec](https
1616
There are 3 main parts for contributing support for a new framework:
1717

1818
- [Test discovery](#test-discovery): identifying tests within the codebase and their structure
19-
- Command resolution: determining how to execute a specific test or group of tests
19+
- [Command resolution](#command-resolution): determining how to execute a specific test or group of tests
2020
- Custom reporting: displaying test execution results in the test explorer
2121

2222
## Test discovery
@@ -220,3 +220,114 @@ module RubyLsp
220220
end
221221
end
222222
```
223+
224+
## Command resolution
225+
226+
Command resolution is the process of receiving a hierarchy of tests selected in the UI and determining the shell commands required to run them. It's important that we minimize the number of these commands, to avoid having to spawn too many Ruby processes.
227+
228+
For example, this is what consolidated commands could look like when trying to run two specific examples in different
229+
frameworks:
230+
231+
```shell
232+
# Rails style execution (very similar to RSpec)
233+
bin/rails test /project/test/models/user_test.rb:10:25
234+
235+
# Test Unit style execution based on regexes
236+
bundle exec ruby -Itest /test/model_test.rb --testcase \"/^ModelTest\\$/\" --name \"/test_something\\$/
237+
238+
# Minitest style execution based on regexes
239+
bundle exec ruby -Itest /test/model_test.rb --name \"/^ModelTest#test_something\\$/\"
240+
```
241+
242+
The add-on's responsibility is to figure out how to execute test items that are associated with the framework they add
243+
support for. Add-ons mark test items with the right framework during discovery and should only resolve items that
244+
belong to them.
245+
246+
Another important point is that test groups with an empty children array are being fully executed. For example:
247+
248+
```ruby
249+
# A test item hierarchy that means: execute the ModelTest#test_something specific example and no other tests
250+
[
251+
{
252+
id: "ModelTest",
253+
uri: "file:///test/model_test.rb",
254+
label: "ModelTest",
255+
range: {
256+
start: { line: 0, character: 0 },
257+
end: { line: 30, character: 3 },
258+
},
259+
tags: ["framework:minitest", "test_group"],
260+
children: [
261+
{
262+
id: "ModelTest#test_something",
263+
uri: "file:///test/model_test.rb",
264+
label: "test_something",
265+
range: {
266+
start: { line: 1, character: 2 },
267+
end: { line: 10, character: 3 },
268+
},
269+
tags: ["framework:minitest"],
270+
children: [],
271+
},
272+
],
273+
},
274+
]
275+
276+
# A test item hierarchy that means: execute the entire ModelTest group with all examples and nested groups inside
277+
[
278+
{
279+
id: "ModelTest",
280+
uri: "file:///test/model_test.rb",
281+
label: "ModelTest",
282+
range: {
283+
start: { line: 0, character: 0 },
284+
end: { line: 30, character: 3 },
285+
},
286+
tags: ["framework:minitest", "test_group"],
287+
children: [],
288+
},
289+
]
290+
```
291+
292+
Add-ons can define the `resolve_test_commands` method to define how to resolve the commands required to execute a
293+
hierarchy. It is the responsibility of the add-on to filter the hierarchy to only the items that are related to them.
294+
295+
```ruby
296+
module RubyLsp
297+
module MyTestFrameworkGem
298+
class Addon < ::RubyLsp::Addon
299+
# Items is the hierarchy of test items to be executed. The return is the list of minimum shell commands required
300+
# to run them
301+
#: (Array[Hash[Symbol, untyped]]) -> Array[String]
302+
def resolve_test_commands(items)
303+
commands = []
304+
queue = items.dup
305+
306+
until queue.empty?
307+
item = queue.shift
308+
tags = Set.new(item[:tags])
309+
next unless tags.include?("framework:my_framework")
310+
311+
children = item[:children]
312+
313+
if tags.include?("test_dir")
314+
# Handle running entire directories
315+
elsif tags.include?("test_file")
316+
# Handle running entire files
317+
elsif tags.include?("test_group")
318+
# Handle running groups
319+
else
320+
# Handle running examples
321+
end
322+
323+
queue.concat(children) unless children.empty?
324+
end
325+
326+
commands
327+
end
328+
end
329+
end
330+
end
331+
```
332+
333+
You can refer to implementation examples for [Minitest and Test Unit](https://github.com/Shopify/ruby-lsp/blob/d86f4d4c567a2a3f8ae6f69caa10e21c4066e23e/lib/ruby_lsp/listeners/test_style.rb#L10) or [Rails](https://github.com/Shopify/ruby-lsp-rails/blob/cb9556d454c8bb20a1a73a99c8deb8788a520007/lib/ruby_lsp/ruby_lsp_rails/rails_test_style.rb#L11).

0 commit comments

Comments
 (0)