detailed test step information in the test result

[EdStrickland]

HI,
our team is currently running BDD and wants to generate a cucumber report. I've written an Karma plugin but found that test steps information weren't in the result.
Is there anyway I could get such information? If not, is there a plan of expossing it?

[matz3]

What do you mean with test steps?

[EdStrickland]

@matz3 I mean the results of all the check points

[matz3]

I'm still not sure what you mean.

This plugin offers two modes.

One is "script" which should not influence the way test results are recorded. This needs to be done in combination with a karma plugin like karma-qunit.

The other one is "html", which has built-in reporting for the QUnit framework, which also means no other test frameworks can be used in this mode. Here it could be that our code doesn't report results back to karma in a proper way, although I'm not aware of any known issues.

Are you able to provide a reproducible example project?

[EdStrickland]

@matz3 I cannot provide our project, so I will need to write a demo. I can only explain the issue as mush as I can while I wrote it.

So when you write a karma-reporter plugin, you can rewrite the callback function onSpecComplete, which receives an object 'result' that gives you information of the test case that has been completed. However, the information doesn't contain which check point the test passed and which isn't.

[matz3]

Got it, thanks 👍
I can have a deeper look into it once a demo/example is provided.

[EdStrickland]

@matz3 I just looked in to the karma-ui5 source code. It seems in 'client/browser.js' the Qunit.log() won't send the result to karma. Is it possible that you provide the result out of the input 'details' and send it to karma?

[matz3]

Currently it is collecting all errors and then posting them within QUnit.testDone to karma.
That's also what the karma-qunit plugin does.

Feel free to submit a PR.

[EdStrickland]

@matz3 Hi, I submitted PR#161 but the pipeline seems not running, is it expected to happen?

[EdStrickland]

#161

[EdStrickland]

Hi,
Is anyone available to help to review my PR? It's been nine days since last updated.

[matz3]

Sorry for the delay. I've added comments to your PR.
But before merging I would still like to fully understand this use case, also in order to properly document it.

[EdStrickland]

Thanks for the review. I'm changing the code accordingly.
About the use case, would you like to know our scenario, or the scenario for the code in general?

[matz3]

I would like to understand how the adopted report is used in combination with cucumber and why those assertions should also be reported as a "test".