Drift checks

[svrooij]

Recently @Ruthhl3ss inspired me to experiment with using Maester for drift control.

❓ Question ❓

Does this test belong in Maester itself?

📃 Description 📃

It works like this:

1. You create a drift folder in the folder next to the tests folder
2. You then add one or more subfolders to it.
3. Each subfolder needs a baseline.json file (which you should check-in!)
4. And before you run the tests you generate or load (or ...) the current.json
5. Each subfolder will be discovered dynamically
6. For each subfolder it will run these 6 tests:
   • Contains baseline.json
   • Contains current.json
   • Are both files valid json
   • Are there keys missing from current that are in baseline
   • Do all properties from baseline.json match with a value in current.json, yes it does a value-by-value compare. So it does not matter if they are in a different order or whether there are whitespaces in the json.

💡 Sample

Created Other and HelloWorld for demo, and this is how it looks.

📷 Folder structure

📷 Report overview

📷 Detailed differences

[soulemike]

Managing state has been a discussion. It is challenging since you could distribute a baseline config, and compare with current, but what is the process for a new or custom baseline. Will Maester only create/overwrite the baseline config if it does not exist, etc...

[svrooij]

I'm not talking about managing state, I was talking about validating two Json files property wise.

The idea is that you check in the baseline, as you would with your custom tests. The current files (that you generate/download/fetch) are fetched right before the test run. Those are compared.

If you want a new baseline file (because you changed something and want to validate your change), you commit a new version of the baseline.

The idea is that you, as the one setting up Maester, decide which files you'll be validating and where they come from. Until you add the files to the "special" folder they do nothing

[SamErde]

I like where this could go. Not sure yet about establishing a baseline to compare to, but what about a simple "here's what has changed since the last run" report? (Compare two Maester results JSON files.)

[svrooij]

That means you need to save it somewhere, surely you can do that on GitHub, but would you then always save and check-in the last version?

And would you just be interested in this test used to fail and is now succeeding. Or also a string compare on the details?

[merill]

Nice. This is a cool way to do generic comparison.

Would be neat to integrate with all the goodness in M365 DSC and either use their raw code to generate an export of json (I think the DSC output right now is not JSON).

Or directly rely on the output from M365DSC

[Ruthhl3ss]

I agree with Stephan. The user must supply the baseline and the comparison themselves. We can't judge what the baseline is (for intune).

Furthermore, unfortunately M365DSC uses their own format not JSON and is reliant on the Microsoft.Graph CMDlets which exports not all values. I think you want to have the full JSON file in order to do a proper compare.