demo the failure issue

[tim-yao]

In my setup(Nightwatch programmatic API with cucumber-js), the Nightwatch API command won't fail the step, even if they do get the error.
Assert and Expect work as expected. If I use try catch, I can catch the error.

See the CI result: https://github.com/nightwatchjs-community/cucumber-nightwatch/actions/runs/3547793604/jobs/5958303663

To reproduce the issue, run

NIGHTWATCH_OUTPUT=true npm run test -- --tags "@fail"

The issue can't be reproduced in https://github.com/nightwatchjs/cucumberjs-boilerplate. So I guess something with my setup but really need some help to debug.

[gravityvi]

Thanks @tim-yao for pointing this out. It's happening because we only reject promises for the waitFor command when the user uses cucumber-js as a test runner: https://github.com/nightwatchjs/nightwatch/blob/main/lib/core/asynctree.js#L95-97. The cucumber-boilerplate project uses cucumber runner in the config and hence the issue will not be reproduced over there. However, I think PR: nightwatchjs/nightwatch#3499 might fix the issue.