Skip to content

Commit 5f9fd1a

Browse files
badeballbadeball
committed
Add a filterSpecsMixedMode option
Filtering non-Cucumber specs (which doesn't contain tags) is not straight forward and there's not a single behavior that's more intuitive than others. Hence a `filterSpecsMixedMode` option. This fixes #1125 [1], essentially reverts 0b2702b [2] by retaining original behavior by default, and also relates to #1116 [3]. [1] #1125 [2] 0b2702b [3] #1116
1 parent 1e3643a commit 5f9fd1a

File tree

9 files changed

+283
-67
lines changed

9 files changed

+283
-67
lines changed

‎CHANGELOG.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,10 @@ All notable changes to this project will be documented in this file.
66

77
- Add order option to all hooks, fixes [#481](https://github.com/badeball/cypress-cucumber-preprocessor/issues/481).
88

9+
- Add a [`filterSpecsMixedMode`](docs/tags.md#tag-filters-and-non-cucumber-specs) option, fixes [#1125](https://github.com/badeball/cypress-cucumber-preprocessor/issues/1125).
10+
11+
- This essentially reverts 0b2702b from v19.1.1 and re-introduces original behavior of discarding non-feature specs by default and introduces an option to control this behavior.
12+
913
## v19.1.1
1014

1115
- Mock and imitate Cypress globals during diagnostics / dry run, fixes [#1120](https://github.com/badeball/cypress-cucumber-preprocessor/issues/1120).

‎docs/configuration.md‎

Lines changed: 13 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -71,18 +71,19 @@ $ CYPRESS_filterSpecs=true cypress run
7171

7272
Every configuration option has a similar key which can be use to override it, shown in the table below.
7373

74-
| JSON path | Environment key | Example(s) |
75-
|--------------------|-------------------|------------------------------------------|
76-
| `stepDefinitions` | `stepDefinitions` | `[filepath].{js,ts}` |
77-
| `messages.enabled` | `messagesEnabled` | `true`, `false` |
78-
| `messages.output` | `messagesOutput` | `cucumber-messages.ndjson` |
79-
| `json.enabled` | `jsonEnabled` | `true`, `false` |
80-
| `json.output` | `jsonOutput` | `cucumber-report.json` |
81-
| `html.enabled` | `htmlEnabled` | `true`, `false` |
82-
| `html.output` | `htmlOutput` | `cucumber-report.html` |
83-
| `pretty.enabled` | `prettyEnabled` | `true`, `false` |
84-
| `filterSpecs` | `filterSpecs` | `true`, `false` |
85-
| `omitFiltered` | `omitFiltered` | `true`, `false` |
74+
| JSON path | Environment key | Example(s) |
75+
|------------------------|------------------------|------------------------------------------|
76+
| `stepDefinitions` | `stepDefinitions` | `[filepath].{js,ts}` |
77+
| `messages.enabled` | `messagesEnabled` | `true`, `false` |
78+
| `messages.output` | `messagesOutput` | `cucumber-messages.ndjson` |
79+
| `json.enabled` | `jsonEnabled` | `true`, `false` |
80+
| `json.output` | `jsonOutput` | `cucumber-report.json` |
81+
| `html.enabled` | `htmlEnabled` | `true`, `false` |
82+
| `html.output` | `htmlOutput` | `cucumber-report.html` |
83+
| `pretty.enabled` | `prettyEnabled` | `true`, `false` |
84+
| `filterSpecsMixedMode` | `filterSpecsMixedMode` | `hide`, `show`, `empty-set` |
85+
| `filterSpecs` | `filterSpecs` | `true`, `false` |
86+
| `omitFiltered` | `omitFiltered` | `true`, `false` |
8687

8788
## Test configuration
8889

‎docs/tags.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,14 @@ Tags are inherited by child elements. Tags that are placed above a `Feature` wil
4141

4242
Normally when running a subset of scenarios using `cypress run --env tags=@foo`, you could potentially encounter files containing no matching scenarios. These can be pre-filtered away by setting `filterSpecs` to `true`, thus saving you execution time.
4343

44+
### Tag filters and non-Cucumber specs
45+
46+
If you are mixing Cucumber and non-Cucumber specs, you can control how non-Cucumber specs are filtered when using `filterSpecs` and tag expressions. Filtering non-Cucumber specs (which doesn't contain tags) is not straight forward and there's not a single behavior that's more intuitive than others. Hence there's a `filterSpecsMixedMode` option. Valid options are:
47+
48+
- "**hide**" (default): non-Cucumber specs are hidden regardless of your tag expression
49+
- "**show**": non-Cucumber specs are shown regardless of your tag expression
50+
- "**empty-set**": non-Cucumber specs are filtered as if having empty set of tags, meaning that positive expressions (`tags=@foo`) will discard these specs, while negative expressions (`tags='not @foo'`) will select them
51+
4452
## Omit filtered tests
4553

4654
By default, all filtered tests are made *pending* using `it.skip` method. If you want to completely omit them, set `omitFiltered` to `true`.

‎features/step_definitions/cli_steps.ts‎

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -74,21 +74,22 @@ Then("it should appear as if both tests were skipped", function () {
7474
);
7575
});
7676

77+
const ranTestExpr = (spec: string) =>
78+
new RegExp("Running:\\s+" + rescape(spec));
79+
80+
Then("it should appear to have ran spec {string}", function (spec) {
81+
assert.match(this.lastRun.stdout, ranTestExpr(spec));
82+
});
83+
7784
Then("it should appear to not have ran spec {string}", function (spec) {
78-
assert.doesNotMatch(
79-
this.lastRun.stdout,
80-
new RegExp("Running:\\s+" + rescape(spec))
81-
);
85+
assert.doesNotMatch(this.lastRun.stdout, ranTestExpr(spec));
8286
});
8387

8488
Then(
8589
"it should appear to have ran spec {string} and {string}",
8690
function (a, b) {
8791
for (const spec of [a, b]) {
88-
assert.match(
89-
this.lastRun.stdout,
90-
new RegExp("Running:\\s+" + rescape(spec))
91-
);
92+
assert.match(this.lastRun.stdout, ranTestExpr(spec));
9293
}
9394
}
9495
);

‎features/tags/spec_filter.feature‎

Lines changed: 79 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -1,73 +1,106 @@
11
Feature: filter spec
22

33
Background:
4-
Given additional preprocessor configuration
4+
Given additional Cypress configuration
5+
"""
6+
{
7+
"e2e": {
8+
"specPattern": "**/*.{spec.js,feature}"
9+
}
10+
}
11+
"""
12+
And additional preprocessor configuration
513
"""
614
{
715
"filterSpecs": true
816
}
917
"""
18+
And a file named "cypress/e2e/foo.feature" with:
19+
"""
20+
@foo
21+
Feature: some feature
22+
Scenario: first scenario
23+
Given a step
24+
"""
25+
And a file named "cypress/e2e/bar.feature" with:
26+
"""
27+
@bar
28+
Feature: some other feature
29+
Scenario: second scenario
30+
Given a step
31+
"""
32+
And a file named "cypress/support/step_definitions/steps.js" with:
33+
"""
34+
const { Given } = require("@badeball/cypress-cucumber-preprocessor");
35+
Given("a step", function() {})
36+
"""
37+
And a file named "cypress/e2e/baz.spec.js" with:
38+
"""
39+
it("should work", () => {});
40+
"""
1041

1142
Rule: it should filter features based on whether they contain a matching scenario
1243

1344
Scenario: 1 / 2 specs matching
14-
Given a file named "cypress/e2e/a.feature" with:
15-
"""
16-
@foo
17-
Feature: some feature
18-
Scenario: first scenario
19-
Given a step
20-
"""
21-
And a file named "cypress/e2e/b.feature" with:
22-
"""
23-
@bar
24-
Feature: some other feature
25-
Scenario: second scenario
26-
Given a step
27-
"""
28-
And a file named "cypress/support/step_definitions/steps.js" with:
29-
"""
30-
const { Given } = require("@badeball/cypress-cucumber-preprocessor");
31-
Given("a step", function() {})
32-
"""
3345
When I run cypress with "--env tags=@foo"
3446
Then it passes
35-
And it should appear to not have ran spec "b.feature"
47+
And it should appear to not have ran spec "bar.feature"
48+
But it should appear to have ran spec "foo.feature"
3649

37-
Rule: non-feature specs should be filtered as if they have tags equalling the empty set
50+
Scenario: 2 / 2 specs matching
51+
When I run cypress with "--env tags=\"@foo or @bar\""
52+
Then it passes
53+
And it should appear to have ran spec "foo.feature" and "bar.feature"
54+
55+
Rule: filterSpecsMixedMode: hide (default) should hide non-feature specs regardless of tag expression
56+
57+
Scenario: positive tag expression
58+
When I run cypress with "--env tags=@foo"
59+
Then it passes
60+
And it should appear to not have ran spec "baz.spec.js"
61+
62+
Scenario: negative tag expression
63+
When I run cypress with "--env \"tags=not @foo\""
64+
Then it passes
65+
And it should appear to not have ran spec "baz.spec.js"
66+
67+
Rule: filterSpecsMixedMode: show should show non-feature specs regardless of tag expression
3868

3969
Background:
40-
Given additional Cypress configuration
70+
Given additional preprocessor configuration
4171
"""
4272
{
43-
"e2e": {
44-
"specPattern": "**/*.{spec.js,feature}"
45-
}
73+
"filterSpecsMixedMode": "show"
4674
}
4775
"""
48-
And a file named "cypress/e2e/a.feature" with:
49-
"""
50-
@bar
51-
Feature: some feature
52-
Scenario: first scenario
53-
Given a step
54-
"""
55-
And a file named "cypress/support/step_definitions/steps.js" with:
56-
"""
57-
const { Given } = require("@badeball/cypress-cucumber-preprocessor");
58-
Given("a step", function() {})
59-
"""
60-
And a file named "cypress/e2e/b.spec.js" with:
76+
77+
Scenario: positive tag expression
78+
When I run cypress with "--env tags=@foo"
79+
Then it passes
80+
And it should appear to have ran spec "baz.spec.js"
81+
82+
Scenario: negative tag expression
83+
When I run cypress with "--env \"tags=not @foo\""
84+
Then it passes
85+
And it should appear to have ran spec "baz.spec.js"
86+
87+
88+
Rule: filterSpecsMixedMode: empty-set should filter non-feature specs as if they have tags equalling the empty set
89+
90+
Background:
91+
Given additional preprocessor configuration
6192
"""
62-
it("should work", () => {});
93+
{
94+
"filterSpecsMixedMode": "empty-set"
95+
}
6396
"""
6497

65-
Scenario: logical not
66-
When I run cypress with "--env 'tags=not @foo'"
98+
Scenario: positive tag expression
99+
When I run cypress with "--env tags=@foo"
67100
Then it passes
68-
And it should appear to have ran spec "a.feature" and "b.spec.js"
101+
And it should appear to not have ran spec "baz.spec.js"
69102

70-
Scenario: not logical not
71-
When I run cypress with "--env tags=@bar"
103+
Scenario: negative tag expression
104+
When I run cypress with "--env \"tags=not @foo\""
72105
Then it passes
73-
And it should appear as if only a single test ran
106+
And it should appear to have ran spec "baz.spec.js"

‎lib/add-cucumber-preprocessor-plugin.ts‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,8 @@ import { getTags } from "./helpers/environment";
4545

4646
import { memoize } from "./helpers/memoize";
4747

48+
import { assertNever } from "./helpers/assertions";
49+
4850
const resolve = memoize(origResolve);
4951

5052
export type AddOptions = {
@@ -137,7 +139,16 @@ export async function addCucumberPreprocessorPlugin(
137139
config as unknown as ICypressConfiguration
138140
).filter((testFile) => {
139141
if (!testFile.endsWith(".feature")) {
140-
return node.evaluate([]);
142+
switch (preprocessor.filterSpecsMixedMode) {
143+
case "hide":
144+
return false;
145+
case "show":
146+
return true;
147+
case "empty-set":
148+
return node.evaluate([]);
149+
default:
150+
assertNever(preprocessor.filterSpecsMixedMode);
151+
}
141152
}
142153

143154
const content = fs.readFileSync(testFile).toString("utf-8");

‎lib/helpers/assertions.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,10 @@ import { createError } from "./error";
22

33
import { isString } from "./type-guards";
44

5+
export function assertNever(value: never): never {
6+
throw new Error("Illegal value: " + value);
7+
}
8+
59
export function fail(message: string) {
610
throw createError(message);
711
}

0 commit comments

Comments
 (0)