Initialising suite data store fails

[akrutscher]

After updating the dotnet plugin from version 0.6.0 to 0.7.2 I get the following Error message when executing a test with gauge:

Initialising suite data store.
failed to initialize suite datastore. Error: Object reference not set to an instance of an object.

The gauge version I use is 1.6.9, html-report 4.3.1. The dotnet plugin version 0.6.0 did not show this error. What may also be of interest is that my test project is targeting net8.0.

Is there anything else I can do to help analyze the problem and possibly fix it?

[chadlwilson]

It might be something else, but can start by checking any reference to Gauge.CSharp.Lib versions inside your project are updated to the latest version (0.11.3).

Otherwise probably need the gauge log file and some more details about your environment to replicate.

[sriv]

there are a few changes like programatic skipping scenarios and support for async/await in tests. I know we had some issues with async before, so that is a possible suspect. If you have a project where I can reproduce the issue, it will help isolating the cause.

[akrutscher]

Hello @chadlwilson , thank you for the quick response! I now updated the Gauge.CSharp.Lib to 0.11.3 as you said, and now I get a stacktrace in the debug log which leads me to where the problem is:

BeforeSuite Hook execution failed : IBM.ISH.Modules.AcceptanceTests.ExecutionHooks.BeforeSuite

  Error Message: Object reference not set to an instance of an object.
  Stacktrace:
     at Gauge.CSharp.Lib.DataStoreExtensions.Get[T](AsyncLocal`1 store, String key)
     at Gauge.CSharp.Lib.ScenarioDataStore.Get[T](String key)
...
     at IBM.ISH.Modules.AcceptanceTests.ExecutionHooks.BeforeSuite() in C:\Users\AndreasKrutscher\dev\ISH-Modules-AT\src\ExecutionHooks.cs:line 44
     at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
     at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

Obviously it is my mistake to access the scenario data store in the BeforeSuite hook. I did not see that before.

I wonder why I thought I did not get the same error with 0.6.0 - probably I did not use the same code in my BeforeSuite execution hook.

[chadlwilson]

So you have a workaround?

Its possibly due to changes in the way the data stores are initialised, made to support async/multithreaded usage.

Possibly the old code just returned nothing when accessed too early, whereas now it expects the store to have been initialised prior. At the very least it might be possible to fail more gracefully in the library.

[akrutscher]

Yes, something has changed that made me take notice. I use methods in BeforeSuite that I also use in BeforeScenario. It would be good to be able to query the state whether the suite data store has been initialized or not. If the suggestion is well received, I would be happy to stay “on line” and be available for tests :-) For now, I'll catch the NullReferenceException and ignore it. Feel free to close the ticket with it.

I am very happy that I can use gauge for our acceptance tests. It works very well for us and saves us a lot of time on manual testing for releases. I would like to say a big thank you for this and for your prompt response :-)

[chadlwilson]

Well, I'm not sure it's a valid use case to be querying the ScenarioDataStore during BeforeSuite - you should be able to factor out the target data store, rather than just catch an error I'd have thought.

On the other hand, I think the user experience here could possibly be considered a bug - or at least rather inconvenient or oddly structured. Confirmed that previously if you accessed it prior to initialization you just got an empty data store.

Now the data stores are initialized in a more structured way (although it does feel clunky from my perspective), so if you try to access a DataStore outside the proper lifecycle it may not be initialized.

[mpekurny]

@chadlwilson I agree with you on both points.

The new system is a clunky. Unfortunately, I was unable to find a better way given the relationship between the plugin and the library. I am open to suggestions, and have ideas for how it could be made better over time, but as you had commented on in my PR, it had already grown way too large as it was.

Second, that while accessing a data store outside it's lifecycle shouldn't be supported, the error message and handling should be improved to make it more clear as to why/where the error is coming from to make a better user experience.