Run and show test results directly in the editor

[sindresorhus]

An awesome idea, from avajs/ava#183, is to run and see the result of a test run directly in the editor. And ability to quickly run the test your cursor is in, in the test file. We could even show the result of tests in the gutter, like linting. Now that we also support watching in AVA, this brings excited possibilities.

Prior art

• https://github.com/jacobmendoza/rspec-tree-runner
• https://github.com/wallabyjs/atom-wallaby
• https://github.com/boogie666/atom-mocha
• https://github.com/gedaiu/tested-atom
• https://github.com/paulpflug/mocha-ui
• https://github.com/TabDigital/atom-mocha-test-runner
• https://github.com/tswaters/atom-mocha-runner
• https://github.com/nguyenchr/atom-cargo-test-runner
• https://github.com/nathansobo/atom-jasmine-runner/
• https://github.com/klorenz/nuclide-test-runner-pytest

Anyone interested in helping out with this?

[jacobmendoza]

Hey @sindresorhus, I'd love to be involved :)

[SamVerschueren]

Happy to help out for VS Code.

[sindresorhus]

@jacobmendoza Awesome! :)

Btw, the prior art section is sorted by interestingness. I really like what you did with rspec-tree-runner.

[sindresorhus]

Block decorations in Atom also bring some exciting possibilities, like showing stack traces for failing tests directly in the test file (with link to jump directly to the source file) and source file.

[sindresorhus]

I'm thinking the first step will be to get AVA running from Atom and just show the result in the statusbar.

@jacobmendoza Do you have any thoughts of how the AVA integration should look like?

[sindresorhus]

@SamVerschueren 😃

[jacobmendoza]

Hey @sindresorhus, thank you!. I really appreciate your comment :).

These are some of the things that I would like to see in the integration. Just random ideas, and I'd love to see feedback. I don't know if they are similar to the things that you have in mind. Also, this is a busy week, so please forgive me if I'm missing something, as I'm writing this without much time to analyse.

1. I agree that the first version should be really simple and iterate over that. Show the result in a panel should be really easy. Regarding showing the output, I don't know if you mean showing just the verbose output of AVA, integrated with the editor (there are plugins for other runners doing basically that) or we are assuming some basic parsing of the TAP output. That should be easy also.
2. I'd like the integration to be able to show some sort of list with all the tests in one panel. Sometimes, is useful when you go into a file with a lot of tests, be able to see a representation of all of them. I know also that there's people who don't really see value in this. Requires parsing the test file and building the structure in Atom. I'm not sure, but it shouldn't be too hard (it would be the same idea under the tree already implemented in rspec-tree-runner).
3. On the other hand, sometimes when doing TDD and performing small iterations for every step, the kind of view that I was describing before is a waste of space. It could be great to be able to change easily to a more compact UI (maybe in the statusbar?) that shows only the number of tests that are passing and failing, or something along these lines.
4. Investigate, as you said about Block decorations in Atom. Very cool.
5. As far as I know (and please correct me if I'm wrong) the default TAP specification does not return the line number. Is it possible to add it to the output (as an extension)?. The line is useful because it can be used as "index" in Atom, and for some interactions, allows you to point the user to the line. I was wondering also now, any plans to support the execution of only one test?.
6. I know this is kind of basic, but I'd like the package to be extremely careful with memory management in Atom. At least in previous versions, it was quite possible to mess things and make the package slower if it was running for long.

What do you think?.

Thank you!

[sindresorhus]

Good ideas!

1. Parsing the TAP output.
2. 👍
3. 👍 I agree, both should be possible. Along the line of our default minimal reporter and --verbose.
4. It should, but noticed now it doesn't anymore: TAP output no longer shows the line that failed ava#569 Assume it will though, in the at field.
5. Agreed, although when it comes to Atom, I'm much more concerned about startup performance and runtime performance than memory usage.