When missing steps are detected, set the test as failed by default instead of pending/inconclusive

[rblemuscriteo]

Reqnroll Version

2.2.1

Which test runner are you using?

NUnit

Test Runner Version Number

4.3.2

.NET Implementation

.NET 8.0

Test Execution Method

Visual Studio Test Explorer

Content of reqnroll.json configuration file

No response

Issue Description

The default behaviour of having a pending state when there is a missing step is giving false positives on Visual Studio and Rider because of it not being considered a test failure.

This is not always detected and needs to add a reqnroll.json file with missingOrPendingStepsOutcome set as Error in every lib that uses reqnroll, which, in a microservice environment, can be tedious.

Even when having only inconclusive tests the higher level are considered successes in both

Visual Studio

and Rider

Would it be possible to change the default behaviour to Error instead of Pending in the default config ?

Reqnroll/Reqnroll/Configuration/ConfigDefaults.cs

Line 16 in c72d3af

public const MissingOrPendingStepsOutcome MissingOrPendingStepsOutcome = Reqnroll.Configuration.MissingOrPendingStepsOutcome.Pending;

Steps to Reproduce

Add a step in a feature file that is not mapped to any steps defined

Link to Repro Project

No response

[Code-Grump]

Can we change the default value? It's not difficult to make that code change. Unfortunately, it would be a breaking change in behaviour; everybody that currently relies on the default would need to change their config.

The crux of the problem seems to be that you'd like to not have to configure this across several projects. If we could provide a way to configure them collectively, you'd be happy?

[rblemuscriteo]

Yes i do agree it's a breaking change but i don't get why having Pending would be the default on missing steps, i'm wondering how many people are adding the config override to avoid false positives on tests

Since we are in a multi repo environment with more and more usages of reqnroll from different teams, an "easy" fix inside a repo on multiple projects could be easy to implement but for multi repositories, other than forking reqnroll, which is risky and will add maintenance cost over ease of use, i don't see an easy solution

We had already 2-3 cases of false positives caused by manual errors

If you have another approach, i'm listening

[304NotModified]

I like the idea, but doubting if this should be a major or minor version bump