With allowRowTests and Scenario outline scenario order is not correct

[SNBogdanov]

Reqnroll Version

3.0.2

Which test runner are you using?

MSTest

Test Runner Version Number

4

.NET Implementation

.NET Framework 4.6.2 or later

Test Execution Method

Visual Studio Test Explorer

Content of reqnroll.json configuration file

{
"$schema": "https://schemas.reqnroll.net/reqnroll-config-latest.json",

"bindingAssemblies": [
],
"runtime": { "stopAtFirstError": true },
"generator": {
"allowDebugGeneratedFiles": false,
"allowRowTests": false
},
"formatters": {
"html": {},
"message": {}
}
}

Issue Description

I have several Scenarios in my feature file.
First scenario is to start application.
After that I have scenario outline to check menu with datatabel of menu items.
In .testrunsettings I set up
false
To preserve order of cases from my feature file
If I set up
"allowRowTests": true
case are running in correct order, but if I set up
"allowRowTests": false
case order is not correct and checking of menus goes before application start
In Attachement is .feature file and trx file from test execution. It is visible that in trx file first test case is not same as first case in feature file

ARN-T5688.zip

Steps to Reproduce

In description all steps are described

Link to Repro Project

No response

[clrudolphi]

In all cucumber-based testing frameworks, Scenarios are always independent of each other. Tests should not rely on execution order in any way.

If application SUT setup is needed before test execution, include that in one of the available BEFORExxx hooks.

[SNBogdanov]

In all cucumber-based testing frameworks, Scenarios are always independent of each other. Tests should not rely on execution order in any way.

This is understandable, but not convinient.
In my particular case I have scenario with more then 100 steps. Some of them are same for other features.
It is not possible to have library of steps to call from. So I managed to organize steps in scenarios. Such scenarios are called like common steps. They are same in all feature files. And they should be run in specific order.
To make this order fixed I set up
OrderTestByNameInClass as false and everything works correctly. The only problem is for Scenario outline with option "allowRowTests": false. I don't know why is this such problem with this settings so it creates wrong call order.

[Code-Grump]

Unfortunately, what you're describing isn't the use-case for Gherkin. Each scenario is intended to be a stand-alone example of some behaviour a system must follow. The choice of using Gherkin is meant to make it easier to map your requirements to tests.

Reqnroll is designed to meet the goals and intended use of the Gherkin language, and although it might look like you should be able to use it as an automation scripting language, that's not something it's intended to cater for.