Skip to content

Commit 12b9a6e

Browse files
nohwndtriptijain2112vinayakmsft
authored
Make default test pattern narrower (#16772)
* Make default test pattern more narrow VSTest2 task specifies a very wide dll matching pattern which includes many unexpected dlls. This includes all TestPlatform dlls, test framework dlls, and often dlls from local nuget package cache. Instead replace it with pattern that matches dlls from bin folder only, and matches the common test dll naming pattern such as: Product.Tests.dll Product.Test.dll ProductUnitTests.dll Product.Unit.Tests.dll But does not match *TestPlatform*.dll or MSTest.TestFramework.dll. This leads to more correct test runs, and faster test runs. Related microsoft/vstest#3939 * Update versions * Update Tasks/VsTestV2/task.loc.json * Fix stars in loc pattern --------- Co-authored-by: triptijain2112 <[email protected]> Co-authored-by: Vinayak <[email protected]>
1 parent b82b1f7 commit 12b9a6e

File tree

3 files changed

+33
-7
lines changed

3 files changed

+33
-7
lines changed

Tasks/VsTestV2/README.md

Lines changed: 29 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,35 @@ Use the following options to select tests and control how the tests are run
1313
* Multiple paths can be specified, one on each line.
1414
* Uses the minimatch patterns. Learn more about minimatch [here](https://aka.ms/minimatchexamples)
1515

16-
For example:
17-
To run tests from any test assembly that has 'test' in the assembly name, `**\*test*.dll`.
18-
To exclude tests in any folder called `obj`, `!**\obj\**`.
16+
Example 1:
17+
Most commonly your test projects follow a naming pattern such as `Product.Tests.dll`, `ProductTests.dll`, `Product.Test.dll`, `Product.UnitTests.dll` or similar. These dlls reside in your `bin` directory. To include all such test dlls use this pattern:
18+
19+
```
20+
**\bin\**\*test.dll
21+
**\bin\**\*tests.dll
22+
```
23+
24+
Example 2:
25+
When it is impossible to determine a naming convention for the tested dlls a wide include pattern can be used (notice the * before .dll). Such pattern can be followed by exclude patterns (starting with `!`) that excludes additional dlls.
26+
This pattern includes all dlls that have `test` in their name, and excludes all dlls from intermediate build `obj` directory:
27+
28+
```
29+
**\*test*.dll
30+
!**\obj\**
31+
```
32+
33+
This pattern is likely to include more dlls than you expect as many other dll names include *test* in their name, such as `MSTest.TestFramework.dll`, `Microsoft.VisualStudio.TestPlatform.ObjectModel.dll` etc. Please review your test log to see which dlls are included, and add appropriate excludes, such as:
34+
35+
```
36+
**\*test*.dll
37+
!**\obj\**
38+
!**\*.resources.dll
39+
!**\*TestAdapter.dll
40+
!**\*Microsoft.*TestPlatform*.dll
41+
!**\*testhost*.dll
42+
!**\testcentric.engine.metadata.dll
43+
```
44+
1945

2046
- **Search Folder:** Use this to specify the folder to search for the test files. Defaults to `$(System.DefaultWorkingDirectory)`
2147

Tasks/VsTestV2/task.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
"author": "Microsoft Corporation",
1818
"version": {
1919
"Major": 2,
20-
"Minor": 220,
20+
"Minor": 224,
2121
"Patch": 0
2222
},
2323
"demands": [
@@ -69,7 +69,7 @@
6969
"name": "testAssemblyVer2",
7070
"type": "multiLine",
7171
"label": "Test files",
72-
"defaultValue": "**\\*test*.dll\n!**\\*TestAdapter.dll\n!**\\obj\\**",
72+
"defaultValue": "**\\bin\\**\\*test.dll\n**\\bin\\**\\*tests.dll",
7373
"required": true,
7474
"helpMarkDown": "Run tests from the specified files.<br>Ordered tests and webtests can be run by specifying the .orderedtest and .webtest files respectively. To run .webtest, Visual Studio 2017 Update 4 or higher is needed. <br><br>The file paths are relative to the search folder. Supports multiple lines of minimatch patterns. [More information](https://aka.ms/minimatchexamples)",
7575
"groupName": "testSelection",

Tasks/VsTestV2/task.loc.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
"author": "Microsoft Corporation",
1818
"version": {
1919
"Major": 2,
20-
"Minor": 220,
20+
"Minor": 224,
2121
"Patch": 0
2222
},
2323
"demands": [
@@ -69,7 +69,7 @@
6969
"name": "testAssemblyVer2",
7070
"type": "multiLine",
7171
"label": "ms-resource:loc.input.label.testAssemblyVer2",
72-
"defaultValue": "**\\*test*.dll\n!**\\*TestAdapter.dll\n!**\\obj\\**",
72+
"defaultValue": "**\\bin\\**\\*test.dll\n**\\bin\\**\\*tests.dll",
7373
"required": true,
7474
"helpMarkDown": "ms-resource:loc.input.help.testAssemblyVer2",
7575
"groupName": "testSelection",

0 commit comments

Comments
 (0)