Should Csla 6 offer a package for another host - unit tests?

[TheCakeMonster]

All of the work that @rockfordlhotka has done to change Csla over to be dependent upon DI for 6.0 might offer an opportunity - or possibly raise a need - for unit testing projects to be a more officially supported hosting environment for Csla.

I'm starting to wonder if there should be another nuget package that is intended to be referenced by test projects. Perhaps we might call it Csla.UnitTesting to fit existing nomenclature. I think what I have put into Csla.TestHelpers is the starting point for [indeed, the majority of the code required for] the specialist part of that package. Renaming Csla.TestHelpers and then bundling it into a nuget package along with Csla.dll is the sort of approach I have in mind.

What excites me most about the change to using DI is that we can make further inroads into dismissing the concerns that some people have about the effect of Csla on the testability of solutions that make use of it. I think Csla gets rejected by a lot of developers unnecessarily because of this worry - I've had quite a number of people tell me that Csla isn't 'modern' for this very reason.

As it happens, I've always felt this testability concern was a bit of a non-issue, because the way I use Csla the BOs themselves almost always don't need testing. The rules they run might, but they are separate classes these days. In addition, I think the layers above them probably don't need to have the BOs mocked out, if you have managed to abstract your data access in some way. Generally, I think people have been misled about what the unit is usually supposed to be in unit testing [spoiler: it's often not a single class, in isolation] and get frightened about testing against a higher layer without abstracting away the BOs - which the original factory method style of coding didn't easily support.

@swegele has raised the question in discussion #2687 about whether there should be samples of unit tests for systems that make use of Csla. I think that's a fair question - but it triggered this slightly meater question when I started to ponder it.

[swegele]

I love the idea!

To your background thoughts, I've also experienced the same, what I would call "dogmatic" pushback, from other's about "unit" tests and Csla. But I've recently completed/delivered an AutomatedTest layer which directly tests my business object layer. I had trouble getting buy-in for this. Originally asked for it back in 2016, but finally got permission and now boss is in love. Now you'd have to pry this test layer from my CEOs cold dead hands :-) Having it is going to be the key for allowing us to release far more frequently and with far more confidence (i.e. cloud deployments)

• It is SICK FAST (for what is being tested - currently runs through twice in ~5 min). I say twice because we can run it once with certain switches off (default legacy behavior) and then again with those switches on since we still support customers that came on with us early 2000 timeframe
• Currently there are over 10K tests and growing every week
• Our DEVOPS test case numbers get marked in the various tests and so our QA staff knows which things they don't need to test anymore
• Tests run as part of CI at Pull Request submission time in hosted Azure Agent
• These are "integration" tests, because at class initialize, I pull the starter db ".bak" via linked file from source control and restore it to LocalDB (stripped down SQL Express). Then the business objects test against real db backend. But what's funny is now that we have them, there are no more arguments about "unit" vs "integration".
• This effectively tests my BL, DAL, and SQL layers! Recently I caught a bug where SQL actually did "case-sensitive" logic (in a BulkInsert operation). It blew our minds that SQL by default did case-sensitive. We caught this bug BEFORE it went out the door because we didn't mock but hit real SQL.

So, we love testing AND Csla. Again, I love your idea.

[rockfordlhotka]

Yes, this sounds like a good idea. Should we spec it out here before creating an issue? Specifically what functionality is desired in this host?

[TheCakeMonster]

I think maybe there are short term and long term objectives:

Short term, I think people will want to make use of the TestDIContextFactory to create custom DI contexts for custom scenarios, from which they can request implementations of IDataPortal<T>. TestDIContextFactory can be used at whatever granularity people want - they could create a single default one for everything, one per test class, or one per test method. As a result of changes made during the fixing up of the old Csla.Test code, this is now much enhanced and more flexible than the single, shared ServiceProvider that I created in the first instance in the Csla.TestHelpers assembly.

Long term, I think there is probably a benefit in creating a custom implementation of IDataPortal<T> that uses a dictionary of pre-created objects to fulfil requests made through the [Inject] attribute.

Using DI to perform tests is a little unusual. It works, but it is not what people expect. They are more used to creating specific instances of classes with predefined data - for example using Moq or similar - and then injecting them themselves - usually via a constructor. A custom implementation of IDataPortal<T> which accepts a list or dictionary of predefined DAL implementations containing custom test data and then provides them to data access methods instead of creating instances via DI would be the way to go to offer what these people expect.

To be clear, I consider there to be two types of test - or maybe two ideas of what a test should be. One test is checking end to end that their usage of CSLA is doing what they are intended, and for this type of test I think we would generally recommend using DI, as this is the closest you can get to real-world execution, making the test more accurate. However, having to register custom classes with DI to return hard-coded data from a DAL is not ideal, and I think the other type of test basically ignores CSLA as much as possible and returns instances of business objects for use within another test fixture using hard-coded DAL classes as simply as possible. I think it's OK to also support this style of testing, even though it doesn't run the same IDataPortal<T> code. That second type of code is about testing a separate fixture, somewhat in isolation, as simply as possible and using predetermined data or behaviour from outside of the fixture.

[rockfordlhotka]

I also wonder if we can help with testing rules, perhaps with some pre-built "mock" context types for business and authorization rules.

[TheCakeMonster]

Trackback to earlier discussions with Jason on this subject, for additional thoughts and context #1736