Add support for Tag Expressions

[clrudolphi]

🤔 What's changed?

This PR adds support for tag expressions on step bindings via the [Scope(Tag = "mytag or thatTag")] syntax and via the attribute argument of a Hook binding, such as [BeforeScenario("mytag or thatTag")].

Compatible with the VisualStudio extension.

⚡️ What's your motivation?

Add tag-expression support bringing closer parity with the other Cucumber implementations.

🏷️ What kind of change is this?

• ⚡ New feature (non-breaking change which adds new behaviour)

♻️ Anything particular you want feedback on?

Any corner cases overlooked?

📋 Checklist:

• I've changed the behaviour of the code
  • I have added/updated tests to cover my changes.
• My change requires a change to the documentation.
  • I have updated the documentation accordingly.
• Users should know about my change
  • I have added an entry to the "[vNext]" section of the CHANGELOG, linking to this pull request & included my GitHub handle to the release contributors list.

---

This text was originally taken from the template of the Cucumber project, then edited by hand. You can modify the template here.

[clrudolphi]

Should we include in this PR the following behavior enhancements:

• via a setting in reqnroll.config, define an Environment variable that will be read on startup that will contain tags that will be inserted into each FeatureInfo.Tags and/or ScenarioInfo.Tags collection? A use case would be to inject the current 'environment' definition (DEV, QA, etc).
• at startup, look for an environment variable (eg, TagFilter) that would contain a tag expression which would be checked at Scenario Start and run only those that evaluate to true. This provides a way for the test script to dynamically filter which tests to run (by tag). This is the equivalent of the tag filtering which Cucumber provides from the command line --tag argument.

[clrudolphi]

@gasparnagy ready for review now that Cucumber.TagExpressions has been published to nuget.

[gasparnagy]

I think this is going to be fine, I have made a few comments that require a bit of rework.

Thinking it over, we should also postpone this to v4, because it might break some tooling.

[gasparnagy]

We should check what happens if someone uses an invalid expression as Tag, e.g. @foo ( @bar. I fear that we need to work extra here to have proper error handling, but I haven't tried it yet.
We should check all the other places where the Parse method is invoked.
We should add a test for this.

[clrudolphi]

The Cucumber.TagExpressionParser will throw a TagExpressionException when invalid tag expression syntax is used.
The only addition I would suggest we add is the custom support for expecting the @ symbol prefix on all terms except for legacy single term expressions.

[gasparnagy]

I was referring to the user experience. If you try the PR code in a simple calculator project (https://github.com/reqnroll/Reqnroll.ExploratoryTestProjects/tree/main/ReqnrollCalculator) with adding an invalid tag expression, like:

    [When("test"), Scope(Tag = "@foo ( @bar")]
    public void WhenTest()
    {
    }

Currently this crashes the VS integration and will report an error, like the one below.

VS error

Warning: AndDiscoveryProviderSucceed: Error during binding discovery. 
Command executed:
  W:\Reqnroll\Reqnroll.ExploratoryTestProjects\ReqnrollCalculator\ReqnrollCalculator.Specs\bin\Debug\net8.0> C:\Program Files\dotnet\dotnet.exe exec c:\users\gaspar\appdata\local\microsoft\visualstudio\18.0_b09b24d6\extensions\hwfrw05j.wgb\Connectors\Reqnroll-Generic-net8.0\reqnroll-vs.dll discovery W:\Reqnroll\Reqnroll.ExploratoryTestProjects\ReqnrollCalculator\ReqnrollCalculator.Specs\bin\Debug\net8.0\ReqnrollCalculator.Specs.dll W:\Reqnroll\Reqnroll.ExploratoryTestProjects\ReqnrollCalculator\ReqnrollCalculator.Specs\reqnroll.json
Exit code: 0
Message: 
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> Cucumber.TagExpressions.TagExpressionException: Tag expression "@core ( @bar" could not be parsed because of syntax error: Expected operator.
   at Cucumber.TagExpressions.TagExpressionParser.ThrowSyntaxError(String message, TagToken tagToken)
   at Cucumber.TagExpressions.TagExpressionParser.ParseTerm()
   at Cucumber.TagExpressions.TagExpressionParser.ParseExpression()
   at Cucumber.TagExpressions.TagExpressionParser.Parse(String text)
   at Reqnroll.Bindings.Discovery.ReqnrollTagExpressionParser.Parse(String tagExpression)
   at Reqnroll.Bindings.Discovery.BindingSourceProcessor.<GetScopes>b__13_1(BindingSourceAttribute attr)
   at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
   at System.Collections.Generic.LargeArrayBuilder`1.AddRange(IEnumerable`1 items)
   at System.Collections.Generic.SparseArrayBuilder`1.ReserveOrAdd(IEnumerable`1 items)
   at System.Linq.Enumerable.Concat2Iterator`1.ToArray()
   at Reqnroll.Bindings.Discovery.BindingSourceProcessor.ProcessMethod(BindingSourceMethod bindingSourceMethod)
   at Reqnroll.Bindings.Discovery.RuntimeBindingRegistryBuilder.BuildBindingsFromType(Type type)
   at Reqnroll.Bindings.Discovery.RuntimeBindingRegistryBuilder.BuildBindingsFromAssembly(Assembly assembly)
   at Reqnroll.Bindings.Provider.BindingProviderService.BuildBindingRegistry(Assembly testAssembly, IRuntimeBindingRegistryBuilder bindingRegistryBuilder)
   at Reqnroll.Bindings.Provider.BindingProviderService.DiscoverBindings(Assembly testAssembly, String jsonConfiguration)
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
   --- End of inner exception stack trace ---
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
   at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
   at ReqnrollConnector.ReflectionExtensions.ReflectionCallStaticMethod[T](Type type, String methodName, Type[] parameterTypes, Object[] args) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\NetExtensions\ReflectionExtensions.cs:line 43
   at ReqnrollConnector.ReqnrollProxies.BindingRegistryFactory.GetBindingRegistry(AssemblyLoadContext assemblyLoadContext, Assembly testAssembly, Option`1 configFile) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\ReqnrollProxies\BindingRegistryFactory.cs:line 17
   at ReqnrollConnector.Discovery.ReqnrollDiscoverer.Discover(IBindingRegistryFactory bindingRegistryFactory, AssemblyLoadContext assemblyLoadContext, Assembly testAssembly, Option`1 configFile) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\Discovery\ReqnrollDiscoverer.cs:line 28
   at ReqnrollConnector.Discovery.DiscoveryCommand.<>c__DisplayClass6_0.<Execute>b__0(IBindingRegistryFactory bindingRegistryFactory) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\Discovery\DiscoveryCommand.cs:line 27
   at FunctionalExtensions.Map[TSource,TResult](TSource this, Func`2 fn) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\NetExtensions\FunctionalExtensions.cs:line 5
   at ReqnrollConnector.Discovery.DiscoveryCommand.Execute(AssemblyLoadContext assemblyLoadContext) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\Discovery\DiscoveryCommand.cs:line 25
   at ReqnrollConnector.ReflectionExecutor.<>c__DisplayClass3_0.<Execute>b__6(DiscoveryCommand cmd) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\ReflectionExecutor.cs:line 84
   at FunctionalExtensions.Map[TSource,TResult](TSource this, Func`2 fn) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\NetExtensions\FunctionalExtensions.cs:line 5
   at ReqnrollConnector.ReflectionExecutor.<>c__DisplayClass3_1.<Execute>b__2() in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\ReflectionExecutor.cs:line 82
   at EitherAdapters.Try[T](Func`1 act) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\NetExtensions\Either\EitherAdapters.cs:line 40
Info Found V3.0.0 at W:\Reqnroll\Reqnroll.ExploratoryTestProjects\ReqnrollCalculator\ReqnrollCalculator.Specs\bin\Debug\net8.0\Reqnroll.dll
Info Chosen BindingRegistryFactoryVLatest for 300000
Error System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> Cucumber.TagExpressions.TagExpressionException: Tag expression "@core ( @bar" could not be parsed because of syntax error: Expected operator.
   at Cucumber.TagExpressions.TagExpressionParser.ThrowSyntaxError(String message, TagToken tagToken)
   at Cucumber.TagExpressions.TagExpressionParser.ParseTerm()
   at Cucumber.TagExpressions.TagExpressionParser.ParseExpression()
   at Cucumber.TagExpressions.TagExpressionParser.Parse(String text)
   at Reqnroll.Bindings.Discovery.ReqnrollTagExpressionParser.Parse(String tagExpression)
   at Reqnroll.Bindings.Discovery.BindingSourceProcessor.<GetScopes>b__13_1(BindingSourceAttribute attr)
   at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
   at System.Collections.Generic.LargeArrayBuilder`1.AddRange(IEnumerable`1 items)
   at System.Collections.Generic.SparseArrayBuilder`1.ReserveOrAdd(IEnumerable`1 items)
   at System.Linq.Enumerable.Concat2Iterator`1.ToArray()
   at Reqnroll.Bindings.Discovery.BindingSourceProcessor.ProcessMethod(BindingSourceMethod bindingSourceMethod)
   at Reqnroll.Bindings.Discovery.RuntimeBindingRegistryBuilder.BuildBindingsFromType(Type type)
   at Reqnroll.Bindings.Discovery.RuntimeBindingRegistryBuilder.BuildBindingsFromAssembly(Assembly assembly)
   at Reqnroll.Bindings.Provider.BindingProviderService.BuildBindingRegistry(Assembly testAssembly, IRuntimeBindingRegistryBuilder bindingRegistryBuilder)
   at Reqnroll.Bindings.Provider.BindingProviderService.DiscoverBindings(Assembly testAssembly, String jsonConfiguration)
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
   --- End of inner exception stack trace ---
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
   at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
   at ReqnrollConnector.ReflectionExtensions.ReflectionCallStaticMethod[T](Type type, String methodName, Type[] parameterTypes, Object[] args) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\NetExtensions\ReflectionExtensions.cs:line 43
   at ReqnrollConnector.ReqnrollProxies.BindingRegistryFactory.GetBindingRegistry(AssemblyLoadContext assemblyLoadContext, Assembly testAssembly, Option`1 configFile) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\ReqnrollProxies\BindingRegistryFactory.cs:line 17
   at ReqnrollConnector.Discovery.ReqnrollDiscoverer.Discover(IBindingRegistryFactory bindingRegistryFactory, AssemblyLoadContext assemblyLoadContext, Assembly testAssembly, Option`1 configFile) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\Discovery\ReqnrollDiscoverer.cs:line 28
   at ReqnrollConnector.Discovery.DiscoveryCommand.<>c__DisplayClass6_0.<Execute>b__0(IBindingRegistryFactory bindingRegistryFactory) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\Discovery\DiscoveryCommand.cs:line 27
   at FunctionalExtensions.Map[TSource,TResult](TSource this, Func`2 fn) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\NetExtensions\FunctionalExtensions.cs:line 5
   at ReqnrollConnector.Discovery.DiscoveryCommand.Execute(AssemblyLoadContext assemblyLoadContext) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\Discovery\DiscoveryCommand.cs:line 25
   at ReqnrollConnector.ReflectionExecutor.<>c__DisplayClass3_0.<Execute>b__6(DiscoveryCommand cmd) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\ReflectionExecutor.cs:line 84
   at FunctionalExtensions.Map[TSource,TResult](TSource this, Func`2 fn) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\NetExtensions\FunctionalExtensions.cs:line 5
   at ReqnrollConnector.ReflectionExecutor.<>c__DisplayClass3_1.<Execute>b__2() in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\ReflectionExecutor.cs:line 82
   at EitherAdapters.Try[T](Func`1 act) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\NetExtensions\Either\EitherAdapters.cs:line 40

The intended behavior should be that the errors are handled and reported to the users. For example if you create an invalid regex:

    [When("^test($")]
    public void WhenTest()
    {
    }

Then you get a nice error and the other step definitions are still working:

In order to handle such errors somehow we would need to ensure that these are returned by the BindingSourceProcessor.ValidateStepDefinition and BindingSourceProcessor.ValidateHook methods. Check the usages of these methods to better understand the concept. Currently this does not work, because we parse the tag expression and fail earlier. Maybe an option would be to be able to represent an "invalid scope" (maybe create a derived type of BindingScope as InvalidBindingScope, that can remember the parser error message. Then in BindingSourceProcessor.ValidateStepDefinition and in BindingSourceProcessor.ValidateHook we can detect the invalid scopes and return a validation error.

[clrudolphi]

I have an approach implemented in which the result of parsing a bad tag expression results in an instance of a new class, InvalidTagExpression : ITagExpression. The BindingScope has been enhanced to surface this expression as a property The BindingSource class has a new method: ValidateBindingScope which checks for this type of tag expression and if found adds an Error BindingValidationResult.
BindingSourceProcessor.ValidateStepDefinition and BindingSourceProcessor.ValidateHook methods call upon the new ValidateBindingScope method.

I am getting strange results.
I have a test (calculator) project with the following additional (5th) binding method added:

    [Given("something"), Scope(Tag = "foo bar")]
    public void GivenAnInvalidScopeTagExpressionIsUsed()
    {
        // This step definition is intentionally left blank.
        // It serves to test the handling of invalid scope tag expressions.
    }

When a test project is run via dotnet test, I get the error messages expected:

   C:\Users\clrud\source\repos\scratch\TagExpressionErrors\TagExpressionErrors\Features\Calculator.feature(10): error TESTERROR:
      Add two numbers (47ms): Error Message: Test method TagExpressionErrors.Features.CalculatorFeature.AddTwoNumbers threw exception:
      Reqnroll.BindingException: Binding error(s) found:
      Tag expression "foo bar" could not be parsed because of syntax error: Expected operator. (at offset 4)

However, in VisualStudio the results don't match the user experience you demonstrated for a faulty regex binding expression.
Output from Reqnroll is:

Note: it indicates 4 imported step definitions. The 5th one is the one with the bad tag expression and it only gives a warning. But, the warning message is not the text expected; I'm not sure where that is coming from.

BUT, the Error List window is empty:

If I execute the test project via the VS Test Explorer, I get a test log that matches what the command line test execution gives.
So I suspect something with the VS Extension. I'm running 2025.3.395 (Reqnroll for Visual Studio 2022 and 2026).

Any suggestions?

[clrudolphi]

And BTW, another oddity; I don't know if this is related.
I have cleared the GeneratedNuGetPackages folder and rebuilt. But I'm seeing version numbers and build dates that don't line up:

Clearing and rebuilding still results in the 3.3.1-local versions coming back, with a file date in the past.

[gasparnagy]

And BTW, another oddity;

Your branch is not updated to v3.3.1-local, therefore your builds produce v3.3.0-local packages. The rebuild does not delete the old packages from the "bin/Debug" folders, but all package files are copied to the GeneratedNuGetPackages. You have built the main branch once already, therefore you have v3.3.1-local versions as well. It's a bit confusing. Probably we should change the script to only copy the packages to the GeneratedNuGetPackages that are related to the currently configured version.

I don't know if this is related.

If in the sample project you used v3.3.0-local then it is not related.

So I suspect something with the VS Extension. I'm running 2025.3.395 (Reqnroll for Visual Studio 2022 and 2026). Any suggestions?

I will check it, but probably only next week will have time for it.

[clrudolphi]

Sorry for being an idiot about the versions.
I will continue investigating how BindingResults are handled within the VS Extension.