index.js: Fallback to require(...) for unrecognized reporters#48
Open
wking wants to merge 7 commits intotapjs:mainfrom
Open
index.js: Fallback to require(...) for unrecognized reporters#48wking wants to merge 7 commits intotapjs:mainfrom
wking wants to merge 7 commits intotapjs:mainfrom
Conversation
Catching up with: * Mocha, which has had these as part of its suites since mochajs/mocha@c0bb9188 (plugging in Hook, 2011-11-25), and * Mochawesome, which has expected arrays since at least adamgruber/mochawesome@5a49d410 (3.0, 2017-11-30, adamgruber/mochawesome#214). Without this change, Mochawesome's: [].concat(suite._beforeAll, suite._beforeEach) will lead to: [undefined, undefined] and Mochawesome will call cleanTest on those undefined values and crash.
d48e15f to
d76f4da
Compare
Catching up with mochajs/mocha@30582e64 (if a reporter has a .done method, call it before exiting, 2014-05-17, mochajs/mocha#1218).
Mochawesome has used: skipped = (!cleaned.pass && !cleaned.fail && !cleaned.pending); since adamgruber/mochawesome@2d4a63b7 (logic to record skipped tests, 2015-01-29). Do not emit 'test end' for skips or TODO (pending) tests, because Mochawesome has: obj.stats.skipped = obj.stats.testsRegistered - obj.stats.tests; since that same commit, so we need these tests entered in testsRegistered (via the 'test' event) but not in tests (via the 'test end' event). Do not emit 'pass' for skips either, otherwise we'll end up with failures + passes > tests. Also fix "pass" -> "passed" for .state. Mochawesome has been comparing the value to "passed" in its cleaner since adamgruber/mochawesome@d687633d (Initial commit, 2014-07-11), and Mocha itself has been comparing with "passed" since at least mochajs/mocha@2c720a35 (do not eat exceptions thrown asynchronously from passed tests, 2018-02-28, mochajs/mocha#3257).
This is, as far as I know, Mochawesome-specific, which makes it a bit of a hack. The Mochawesome side of this is adamgruber/mochawesome@65e90621 (implement addContext method for adding report context to tests, 2016-12-20, adamgruber/mochawesome#106). Ideally we'd want a way to trigger this as part of requesting Mochawesome as the reporter, but the addition seems harmless enough so I'm just including it every time. In the diag.source case, we might want to expose diagnostics besides .fn. But we surely don't want to expose the test source via both .fn and .context. For this commit, I've left the diag.source case alone, but in future work we might want something closer to : if (result.diag) { var context = Object.assign({}, result.diag) if (context.source) { var source = context.source delete context.source this.fn = { toString: function () { return 'function(){' + source + '\n}' } } } if (Object.keys(context).length) { this.context = { title: 'diagnostic', value: context, } } } I haven't done that here, because Object.assign is not supported on Internet Exporer, Android webview, or Opera for Android [1], despite being part of ECMAScript 2015 [2]. Object.keys seems to be supported everywhere [3]. We may only care about Node for this package, but I'm being conservative for this commit. [1]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/assign#Browser_compatibility [2]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/assign#Specifications [3]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/keys#Browser_compatibility
Skips like [1]:
ok 23 # skip Insufficient flogiston pressure.
are parsed (at least by tap-parser@1.2.2) into structures like:
{
id: 1,
name: "",
ok: true,
skip: "root.readonly is false but the root filesystem is still not writable"
}
But title-less tests are not very useful. With this change, cases
like the above empty-string name will fall back to the skip value.
And if that skip value is undefined, we'll fall back to an empty
string (because I'm not sure how well downstream consumers would
handle and undefined title).
One positive effect of this change is that Mochawesome now has a title
message to render for these skips (where previously it just used an
empty h4).
[1]: https://testanything.org/tap-version-13-specification.html#skipping-tests
Mochawesome has expected a suite attached to the runner since adamgruber/mochawesome@e25a5eb0 (handle nested describes, 2014-07-14). This commit creates a new root Suite for that, and attaches the per-TAP suites as children of the root suite. This will allow us to render output from multiple TAP files with Mochawesome. I've also refactored the ancestory properties: * Test.parent became Test.suite, since that's the only object type we're using there. * Suite.parent now only references other Suites. There's no need to complicate the ancestry by including both Suites and Parsers, and sticking to suites means we don't need Parser methods for title chaining. The new titlePath methods catch us up with mochajs/mocha@aa24e82f (indent test contexts, 2017-05-20, mochajs/mocha#2814). I've left fullTitle around for now for compat with older versions of Mocha.
This allows us to use reporters that aren't built in. For example, with this commit you can: $ npm install $ npm install --no-save mocha mochawesome $ cp index.js node_modules/tap-mocha-reporter/ $ cp lib/*.js node_modules/tap-mocha-reporter/lib/ $ npm test -- --reporter=mochawesome That's installing mocha@5.2.0 and mochawesome@3.0.2.
d76f4da to
e4f8183
Compare
Author
|
I've pushed a number of other commits to this branch to get Mochawesome working. With the current tip (e4f8183), you can test with two dummy TAP generators: $ cat /tmp/test1.sh
#!/bin/sh
cat <<EOF
TAP version 13
ok 1 # SKIP testing a skip
not ok 2 - can multiply one by two
---
{
"actual": 3,
"expected": 2
}
...
ok 3 - can multiply two by two
---
{
"actual": 4,
"expected": 4
}
...
1..3
EOF
$ cat /tmp/test2.sh
#!/bin/sh
cat <<EOF
TAP Version 13
1..2
# nesting
1..2
ok 1 - true is ok
not ok 2 - doag is also okay
not ok 1 - nesting
ok 2 - second
EOFInstall $ npm install
$ npm install --no-save mocha mochawesome
$ cp index.js node_modules/tap-mocha-reporter/
$ cp lib/*.js node_modules/tap-mocha-reporter/lib/Test this repository, feeding the output into Mochawesome: $ npm test -- --reporter=mochawesomeTest the dummy generators, feeding the output into Mochawesome: $ node_modules/.bin/tap --reporter=mochawesome /tmp/test1.sh /tmp/test2.shFor folks who are curious, but not curious enough to run the above themselves ;), you can see the generated output here while this PR is in flight. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This allows us to use reporters that aren't built in. For example, with this commit you can:
Fixes #46.