[MTP] Filter string includes unrecognized escape sequence

[cbersch]

The following tests still fail with Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.TestPlatformFormatException: Filter string 'QuotesTests.Test\("C:\\\Path\\\File.txt\\""\)' includes unrecognized escape sequence. when hitting "Run all" in test explorer.

Tested with with 6.0.0-alpha.100 (nunit/nunit3-vs-adapter.issues#10)

If any of the two test cases are commented out, it works fine:

public sealed class QuotesTests
{
    [TestCase("\"C:\\Path\\File.txt\"")]
    [TestCase("C:\\Path\\File.txt\"")]
    public void Test(string input) { }
}

Full stacktrace

========== Starting test run ==========
Connecting to client host '127.0.0.1' port '55833'
NUnit Adapter 6.0.0.0: Test execution started
Process: C:\Temp\mtp_exception\bin\Debug\net10.0\MyProject.dll, Exception: StreamJsonRpc.RemoteInvocationException: Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.TestPlatformFormatException: Filter string 'QuotesTests.Test\("C:\\\Path\\\File.txt\\""\)' includes unrecognized escape sequence.
   at Microsoft.Testing.Extensions.VSTestBridge.ObjectModel.ContextAdapterBase.GetTestCaseFilter(IEnumerable`1 supportedProperties, Func`2 propertyProvider) in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/ObjectModel/ContextAdapterBase.cs:line 53
   at NUnit.VisualStudio.TestAdapter.VsTestFilter.get_MsTestCaseFilterExpression() in D:\a\nunit3-vs-adapter\nunit3-vs-adapter\src\NUnitTestAdapter\VsTestFilter.cs:line 92
   at NUnit.VisualStudio.TestAdapter.VsTestFilter.get_IsEmpty() in D:\a\nunit3-vs-adapter\nunit3-vs-adapter\src\NUnitTestAdapter\VsTestFilter.cs:line 94
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.InitializeForExecution(IRunContext runContext, IFrameworkHandle frameworkHandle) in D:\a\nunit3-vs-adapter\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 286
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle) in D:\a\nunit3-vs-adapter\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 124
   at NUnit.VisualStudio.TestAdapter.TestingPlatformAdapter.NUnitBridgedTestFramework.SynchronizedRunTestsAsync(VSTestRunTestExecutionRequest request, IMessageBus messageBus, CancellationToken cancellationToken) in D:\a\nunit3-vs-adapter\nunit3-vs-adapter\src\NUnitTestAdapter\TestingPlatformAdapter\NUnitBridgedTestFramework.cs:line 42
   at Microsoft.Testing.Extensions.VSTestBridge.SynchronizedSingleSessionVSTestBridgedTestFramework.<>c__DisplayClass22_0.<<ExecuteRequestAsync>b__0>d.MoveNext() in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/SynchronizedSingleSessionVSTestAndTestAnywhereAdapter.cs:line 166
--- End of stack trace from previous location ---
   at Microsoft.Testing.Extensions.VSTestBridge.SynchronizedSingleSessionVSTestBridgedTestFramework.ExecuteRequestWithRequestCountGuardAsync(Func`1 asyncFunc) in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/SynchronizedSingleSessionVSTestAndTestAnywhereAdapter.cs:line 188
   at Microsoft.Testing.Extensions.VSTestBridge.VSTestBridgedTestFrameworkBase.ExecuteRequestAsync(ExecuteRequestContext context) in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/VSTestBridgedTestFrameworkBase.cs:line 93
   at Microsoft.Testing.Platform.Requests.TestHostTestFrameworkInvoker.ExecuteRequestAsync(ITestFramework testFramework, TestExecutionRequest request, IMessageBus messageBus, CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Requests/TestHostTestFrameworkInvoker.cs:line 72
   at Microsoft.Testing.Platform.Requests.TestHostTestFrameworkInvoker.ExecuteAsync(ITestFramework testFramework, ClientInfo client, CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Requests/TestHostTestFrameworkInvoker.cs:line 61
   at Microsoft.Testing.Platform.Hosts.CommonHost.ExecuteRequestAsync(ProxyOutputDevice outputDevice, ITestSessionContext testSessionInfo, ServiceProvider serviceProvider, BaseMessageBus baseMessageBus, ITestFramework testFramework, ClientInfo client) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/CommonTestHost.cs:line 141
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.ExecuteRequestAsync(RequestArgsBase args, String method, ServiceProvider perRequestServiceProvider, CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 498
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.ExecuteRequestAsync(RequestArgsBase args, String method, ServiceProvider perRequestServiceProvider, CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 531
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.HandleRequestCoreAsync(RequestMessage message, RpcInvocationState rpcInvocationState, CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 424
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.HandleRequestAsync(RequestMessage request, CancellationToken serverClosing, CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 330
   at StreamJsonRpc.JsonRpc.InvokeCoreAsync[TResult](RequestId id, String targetName, IReadOnlyList`1 arguments, IReadOnlyList`1 positionalArgumentDeclaredTypes, IReadOnlyDictionary`2 namedArgumentDeclaredTypes, CancellationToken cancellationToken, Boolean isParameterObject)
   at Microsoft.VisualStudio.TestWindow.Internal.RemoteAgent.TestingPlatform.RpcClient.RunTestsWithFilterAsync(Guid runId, TestNode[] testNodes, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Threading.ThreadingTools.WithCancellationSlow[T](Task`1 task, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.TestWindow.Internal.RemoteAgent.TestingPlatform.Clients.TestingPlatformClient.ExecuteRpcCallAsync[T](Func`1 rpcFunction, CancellationToken cancellationToken)
RPC server exception:

[davidknapman-cl]

We're also hitting this same problem on a whole slew of our tests that have various XPath expressions in our test cases. This is with the released 6.0.0 version, not the preview releases.

[Grevor]

Can also confirm that we've hit this hard (around 200 test projects cannot be run at all locally - thankfully seem to work fine in CI/CD). Due to company policy, I sadly cannot share the test names nor input. Version 6.0.0, only an issue under MTP.

The double backslashes in a string/test name together with at least two TestCase or TestCaseData given to a single test function seem to be the trigger(?).

I'll try to give more failure examples tomorrow if I can.

[MaceWindu]

// works without MTP
// cannot be run either explicitly or implicitly with MTP2
[Test]
public void Test1([Values("|")] string arg)
{
    Assert.Pass();
}

// works without MTP
// works when run explicitly(!) in MTP2
// gives error when run implicitly in MTP2
[Test]
public void Test2([Values("\\")] string arg)
{
    Assert.Pass();
}

error for test2

Process: MTP2.dll, Exception: StreamJsonRpc.RemoteInvocationException: Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.TestPlatformFormatException: Filter string 'Tests.Test2\("\\\"\)' includes unrecognized escape sequence.
   at Microsoft.Testing.Extensions.VSTestBridge.ObjectModel.ContextAdapterBase.GetTestCaseFilter(IEnumerable`1 supportedProperties, Func`2 propertyProvider) in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/ObjectModel/ContextAdapterBase.cs:line 53
   at NUnit.VisualStudio.TestAdapter.VsTestFilter.get_MsTestCaseFilterExpression() in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\VsTestFilter.cs:line 92
   at NUnit.VisualStudio.TestAdapter.VsTestFilter.get_IsEmpty() in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\VsTestFilter.cs:line 94
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.InitializeForExecution(IRunContext runContext, IFrameworkHandle frameworkHandle) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 290
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 126
   at NUnit.VisualStudio.TestAdapter.TestingPlatformAdapter.NUnitBridgedTestFramework.SynchronizedRunTestsAsync(VSTestRunTestExecutionRequest request, IMessageBus messageBus, CancellationToken cancellationToken) in C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\TestingPlatformAdapter\NUnitBridgedTestFramework.cs:line 42
   at Microsoft.Testing.Extensions.VSTestBridge.SynchronizedSingleSessionVSTestBridgedTestFramework.<>c__DisplayClass22_0.<<ExecuteRequestAsync>b__0>d.MoveNext() in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/SynchronizedSingleSessionVSTestAndTestAnywhereAdapter.cs:line 166
--- End of stack trace from previous location ---
   at Microsoft.Testing.Extensions.VSTestBridge.SynchronizedSingleSessionVSTestBridgedTestFramework.ExecuteRequestWithRequestCountGuardAsync(Func`1 asyncFunc) in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/SynchronizedSingleSessionVSTestAndTestAnywhereAdapter.cs:line 188
   at Microsoft.Testing.Extensions.VSTestBridge.VSTestBridgedTestFrameworkBase.ExecuteRequestAsync(ExecuteRequestContext context) in /_/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/VSTestBridgedTestFrameworkBase.cs:line 93
   at Microsoft.Testing.Platform.Requests.TestHostTestFrameworkInvoker.ExecuteRequestAsync(ITestFramework testFramework, TestExecutionRequest request, IMessageBus messageBus, CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Requests/TestHostTestFrameworkInvoker.cs:line 72
   at Microsoft.Testing.Platform.Requests.TestHostTestFrameworkInvoker.ExecuteAsync(ITestFramework testFramework, ClientInfo client, CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Requests/TestHostTestFrameworkInvoker.cs:line 61
   at Microsoft.Testing.Platform.Hosts.CommonHost.ExecuteRequestAsync(ProxyOutputDevice outputDevice, ITestSessionContext testSessionInfo, ServiceProvider serviceProvider, BaseMessageBus baseMessageBus, ITestFramework testFramework, ClientInfo client) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/CommonTestHost.cs:line 141
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.ExecuteRequestAsync(RequestArgsBase args, String method, ServiceProvider perRequestServiceProvider, CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 498
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.ExecuteRequestAsync(RequestArgsBase args, String method, ServiceProvider perRequestServiceProvider, CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 531
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.HandleRequestCoreAsync(RequestMessage message, RpcInvocationState rpcInvocationState, CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 424
   at Microsoft.Testing.Platform.Hosts.ServerTestHost.HandleRequestAsync(RequestMessage request, CancellationToken serverClosing, CancellationToken cancellationToken) in /_/src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.cs:line 330
   at StreamJsonRpc.JsonRpc.InvokeCoreAsync[TResult](RequestId id, String targetName, IReadOnlyList`1 arguments, IReadOnlyList`1 positionalArgumentDeclaredTypes, IReadOnlyDictionary`2 namedArgumentDeclaredTypes, CancellationToken cancellationToken, Boolean isParameterObject)
   at Microsoft.VisualStudio.TestWindow.Internal.RemoteAgent.TestingPlatform.RpcClient.RunTestsWithFilterAsync(Guid runId, TestNode[] testNodes, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Threading.ThreadingTools.WithCancellationSlow[T](Task`1 task, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.TestWindow.Internal.RemoteAgent.TestingPlatform.Clients.TestingPlatformClient.ExecuteRpcCallAsync[T](Func`1 rpcFunction, CancellationToken cancellationToken)

[MaceWindu]

And running test exe directly:

• no filters - both tests run
• MTP2.exe --filter Name~Test1\(\"\|\"\) works (" needs escaping for command line)
• MTP2.exe --filter Name~Test2\(\"\\\"\) : TestPlatformFormatException: Filter string 'Test2\("\"\)' includes unrecognized escape sequence but should work according to escaping rules
• MTP2.exe --filter Name~Test2\(\"\\\\"\) : crash with OOM exception in several seconds
• MTP2.exe --filter Name~Test2\(\"\\\\\"\) : doesn't find any tests as expected but also doesn't crash