add examples to prove retry and fail fast interplay (cucumber#1567)

* add examples to prove retry and fail fast interplay

* fix name

* update cli docs

Author: davidjgoss

docs/cli.md

@@ -153,6 +153,8 @@ abort the run on first failure (default: false)
 
 By default, cucumber-js runs the entire suite and reports all the failures. This flag allows a developer workflow where you work on one failure at a time. Combining this feature with rerun files allows you to work through all failures in an efficient manner.
 
+A note on using in conjunction with `--retry`: we consider a test case to have failed if it exhausts retries and still fails, but passed if it passes on a retry having failed previous attempts, so `--fail-fast` does still allow retries to happen.
+
 ## Retry failing tests
 
 Use `--retry <int>` to rerun tests that have been failing. This can be very helpful for flaky tests.

features/retry.feature

@@ -534,3 +534,60 @@ Feature: Retry flaky tests
       """
     When I run cucumber-js with `--retry 1`
     Then it passes
+
+  Rule: using retry in combination with fail-fast will exhaust retries before failing test run
+
+    Scenario: a flaky scenario that passes on the second attempt, set to fail fast
+      Given a file named "features/a.feature" with:
+      """
+      Feature:
+        Scenario: Flaky
+          Given a flaky step
+
+        Scenario: Passing
+          Given a passing step
+      """
+      Given a file named "features/step_definitions/cucumber_steps.js" with:
+      """
+      const {Given} = require('@cucumber/cucumber')
+
+      let willPass = false
+
+      Given(/^a flaky step$/, function() {
+        if (willPass) {
+          return
+        }
+        willPass = true
+        throw 'fail'
+      })
+
+      Given(/^a passing step$/, function() {})
+      """
+      When I run cucumber-js with `--retry 1 --fail-fast`
+      Then it passes
+      And scenario "Flaky" attempt 0 step "Given a flaky step" has status "failed"
+      And scenario "Flaky" attempt 1 step "Given a flaky step" has status "passed"
+      And scenario "Passing" step "Given a passing step" has status "passed"
+
+    Scenario: a scenario that fails every allotted attempt, set to fail fast
+      Given a file named "features/a.feature" with:
+      """
+      Feature:
+        Scenario: Failing
+          Given a failing step
+
+        Scenario: Passing
+          Given a passing step
+      """
+      Given a file named "features/step_definitions/cucumber_steps.js" with:
+      """
+      const {Given} = require('@cucumber/cucumber')
+
+      Given(/^a failing step$/, function() { throw 'fail' })
+      Given(/^a passing step$/, function() {})
+      """
+      When I run cucumber-js with `--retry 1 --fail-fast`
+      Then it fails
+      And scenario "Failing" attempt 0 step "Given a failing step" has status "failed"
+      And scenario "Failing" attempt 1 step "Given a failing step" has status "failed"
+      And scenario "Passing" step "Given a passing step" has status "skipped"