Clarification on DSC v3 for Custom Azure Machine Configuration Packages and Azure Arc IaC Best Practices

[boaz-raz]

Hi everyone,

I'm exploring the best ways to leverage DSC v3 for managing machine configurations for servers connected via Azure Arc, following an Infrastructure as Code (IaC) approach.

My understanding from the documentation (like How to create custom machine configuration package artifacts) is that creating custom machine configuration package artifacts is the recommended method for deploying configurations, including those using DSC.

I have a few specific questions regarding DSC v3 in this context:

1. DSC v3 for Custom Packages: Can we confirm that the process outlined in the linked Microsoft Learn article for creating custom machine configuration packages (developing the .mof, packaging with New-GuestConfigurationPackage, etc.) is fully applicable and the recommended approach when authoring configurations using DSC v3 resources and paradigms? Are there any DSC v3-specific considerations or best practices we should be aware of during this package creation and deployment process, particularly for both Windows and Linux nodes managed by Azure Arc?

2. DSC v3 Features in Machine Configuration: With the advancements in DSC v3 (e.g., new resource manifest, JSON/YAML configurations, potential Bicep integration for CaC), how are these features best utilized when creating and deploying custom machine configuration packages? For instance:

   • Are there updated tools or cmdlets beyond New-GuestConfigurationPackage that are tailored for DSC v3 constructs?
   • What is the current status or roadmap for using Bicep to define DSC v3 configurations that are then packaged for Azure Machine Configuration?

3. Alternative IaC Approaches with DSC v3 and Azure Arc: Besides creating custom .zip packages and deploying them via Azure Policy, are there other recommended or emerging IaC patterns for managing machine state with DSC v3 on Azure Arc-enabled servers? For example, direct application of DSC v3 configurations through other Azure services or CLI commands that align with IaC principles?

I'm keen to ensure we're aligning with the latest capabilities and recommended practices for using DSC v3 with Azure Machine Configuration and Azure Arc. Any insights, examples, or roadmap information would be greatly appreciated!

Thanks!

[Gijsreyn]

H @boaz-raz, the quick answer on all 3 questions: Machine Configuration hasn't implemented DSC v3 (yet). As far as I know, there are some talks about it, but no clear roadmap. Not to say that yes, I would also like to see this :)

[mimachniak]

I think before this implementation, we need to add auto-resource installation of DSC, as now we are working on cache there this is next step.

[boaz-raz]

@Gijsreyn @mimachniak

Thanks for the quick response and clarifying that Azure Machine Configuration doesn't yet support DSC v3. That definitely shapes the current approach for managing Arc-enabled servers.

Given this, and since I'm new to DSC, what's the recommended way to apply DSC v3 as Infrastructure as Code (IaC) for on-premise servers right now?

I was thinking of writing a small PowerShell script to act as a local "agent." This agent would regularly check a UNC path for DSC v3 YAML configuration files, then perform dsc test to see if the server is in the desired state. If not, it would execute dsc set to apply the configuration. (I have a small PoC working for this.)

Would this "agent" approach be a viable and recommended path for immediate on-premise DSC v3 IaC, or are there other patterns or tools you'd suggest considering in the interim? Any feedback on this approach would be greatly appreciated!

[michaeltlombardi]

If I was starting from scratch for on-premises IaC without using any higher order tools, here's the approach I would probably look at taking for experimenting with DSC for managing my servers:

Phase 1: Initializing

1. Drafting - I would write a configuration document to manage a small number of related settings, like setting registry keys.

2. Testing - I would write a small set of acceptance tests for the desired state. Having been around the configuration management space for a long time, I would very strongly recommend having a secondary tool to validate your state outside of your configuration management tool. Any CM tool, including DSC, Puppet, Chef, Ansible, Azure Machine Configuration, etc will faithfully report the state as the resource reports it. If there's a bug in the resource implementation, you might apply a configuration repeatedly and never understand that anything is wrong.

3. Documenting: I would write (minimal) documentation for the configuration document - the document itself tells us "what" and the resources handle the "how", but only you know the "why". It doesn't need to be exhaustive, but adding information about the purpose of the document now will help enormously in the future.

4. Committing: I would commit both the configuration document and tests to a repository, so I can get all the benefits of treating my infrastructure definitions as code. I would use whatever code repository my organization is already using - preferably one where I can track issues and cross-reference other projects. This will also enable me to get a historical state of the configuration document, which can be very useful for onboarding team members, talking with auditors, and so on.

5. Scripting: I would consider creating a small PowerShell module for managing this configuration document with an eye towards maintainability and future expansion. I would probably want to be able to use the functions and any scripts in a continuous integration (CI) context as well as for interactive operations. I might write functions like:

   • Get-ConfigurationDocument - retrieve a configuration document locally or from the repository (or artifact feed)
   • Install-DscCli - Retrieve DSC and install on a machine, depending on how my organization wants software to get to our machines. For example, I could have the command retrieve the published artifact from GitHub, extract it, unblock the files, and ensure the folder is added to the PATH. Alternatively, I could pull DSC from a software package manager repository we're already using. I might even implement this function to operate against remote machines as well as local.
   • Initialize-Dsc - Setup a machine for running DSC as a scheduled task. I'd probably implement it to reuse the Install-DscCli function to get DSC on the machine if it's not already there. Personally I would probably use a .psd1 (or JSON) file to control the schedule and serve as parameters for the DscAgent.ps1 script.

   These are just a few quick examples, what I would actually do depends heavily on the context I'm working within. The point for me would be to make sure I'm developing a maintainable set of tools I can use and build on as we bring more nodes and settings under configuration management.

Phase 2: The first server

I would look to get a single machine under control first and see how things go for a week or so. I'd ask myself and my teammates questions like:

• What's the next setting we should add to this configuration document?
• Are we getting any value from the output? Is there anything we're missing?
• How can we enhance this? Should we tie the scheduled task into any dashboards, log collection, or other tools?
• Is this having any impact on operations? How much friction do we have in our process? Can we reduce that friction?
• When we update the configuration document, how ergonomic is that process?
• Which other servers should we prioritize bringing under control?

I would be careful to remind myself and the team that whatever we've built or done to this point should be considered a prototype and be willing to pivot any component or implementation to suit our needs. We're in the process of building custom tooling to manage our infrastructure and we should always keep in mind who the tools are for and what purposes are most important to us.

Phase 3: Adding Complexity

After a week or two of managing that first server with a small configuration document, I would work with the team on either bringing other servers under control, defining an additional document, or expanding the scope of that document. I wouldn't try to do all of these things at once. Maybe we can start thinking about how we want to deploy the script agent/scheduled task in more detail. Maybe we start to consider a model for representing our environment and how to incorporate that - should we have multiple baselines? Should we use parameters? Should we pre-process configuration documents to go from generic to specific?

Thoughts

Configuration management tooling is a hugely complex space. My go-to advice for everyone for the last decade has been: if you don't have to try building your own tooling from the ground up, don't. If anyone in your organization is already using a higher order configuration management tool, start working with them to see whether and how you can collaborate. If those tools don't yet support DSC, starting the conversation and showing a prototype can help you move forward.

I usually describe the complexity of configuration management as "exploding exponential complexity" - the more servers you need to manage, the more settings you want to control, the more you're going to find yourself wishing you had abstractions and helper tools to ease that problem as it grows by orders of magnitude.

But I've also worked in shops where there wasn't any CM in place and getting a higher order tool was a long uphill battle, so we built some internal tools to learn and bridge the gaps. I can definitely see a way forward for doing that, but it's always going to be contextual and the most important thing to me in this process has never been the specific tools. It's the change in approach and thinking, moving towards declarative idempotent modelling of our infrastructure and seeing how we make it serve the needs of ourselves, our partner teams, our users, and our business. It's a great opportunity to reexamine how we've been managing our infrastructure and choose new models and modes for that work.

[haodeon]

I'm currently trialing DSC v3 in production.

I think DSC v3 is a good configuration management engine but it's a "no batteries included" approach. Some view the lack of orchestration as a downside, I think this the best thing DSC has ever done.

The approach I took was

1. Create a pipeline for automating a bootstrap package.
2. Bootstrap DSC v3 idempotently
3. Bundle PSDSC dependencies as a local nuget repo
4. Use a GitOps approach to configuration documents, I used a rendered manifest monorepo.

There's probably other engineering pain points I've forgotten about and there is still more to do for a comprehensive solution.

[haodeon]

I’ve open-sourced an early version of the pattern I described:
👉 haodeon/dsc-example

What’s included so far:

• A pipeline that builds a bootstrap package
• The full bootstrap automation script, including DSC v3 + PS dependencies
• A local NuGet-based module bundling approach

I plan to expand it with more examples of rendered manifest flows and configuration document handling.

[haodeon]

I've added some static configuration documents to demonstrate structure and composition.

[PshMike]

I too have started down the road of trying to build a process to use DSCv3 in my environment. I currently have a pilot of about 100 servers test servers ( out of my total server herd of about 4000 ).

Some lessons I've learned in the past few weeks while working on this ...

1. "Start with the end in mind." -> If you are interested in DSC, you likely are very interested in knowing "Do I have any drift right now?" In order to answer that question, you will need some means of reporting, and in order to report on things you will need some way to log results and collect those logs. I write events to a custom Windows Event Log and then use Azure Monitor to collect that log data. Of course one will need to first get over the hurdle of actually performing a TEST or SET, but my point is don't leave reporting to the end or you may find your logging along the way was inadequate.

2. The actual DSC work is easy, it's the surrounding stuff that will take up your time. My error handling and logging code greatly outnumbers the code actually calling DSC.EXE. I may be doing something wrong, but I've found if I do a TEST and I have drift that drift won't specified in the result of the TEST pass, so I do an additional GET and then process the output to find exactly what was different.

3. Plan for mistakes. In my environment I can't afford to have a "bad config" get pushed out AND applied to servers. That would be inexcusable. What I can tolerate is a bad config being pushed out and a large portion of the herd reporting they are out of compliance. My point is even using DSC I will need to adhere to strict change management, and that means DSC running on a schedule won't ever perform a SET operation on its own. That will only happen if I set a flag on the server saying it is in a change window and allowed to do a SET. That flag file is managed by a process tied to change management, not DSC.

4. Full vs. Partial Configs ( my weird terminology ) -> My server herd is diverse. Some of my security-related settings are done in a "Servers Built after 2024-01-01" type of baseline. Most settings are not "back-ported" to existing servers for fear of breaking an application. I'm experimenting with authoring configuration documents that apply to clients that meet criteria that is evaluated on the endpoint. For instance, a specific DSC configuration YAML file that will apply to an endpoint if a CIM Query returns data ( "SELECT * from Win32_OperatingSystem WHERE installdate > '01/01/2024' " ).

Put another way, I don't want to have a repo of 4000 configuration documents, one for each server. Instead, I'll have a repo with a number of small independent configuration documents that are focused on one area ( local firewall, security, roles, etc ) and the endpoint will need to determine which configurations need to be evaluated. This makes logging even more important, as one will want to know "why did this server not apply this config?"

Does this approach sound crazy? Am I on the road to heartbreak?

I'm not 100% committed to this method, so any feedback is appreciated.

[haodeon]

@PshMike The main challenge I would highlight is, how does one understand what is the desired state of a server? They would need to understand all the roles and the association of each role to each server. The state of the server is implicit and defined at runtime.

[Borgquite]

@PshMike The only thing I'd say (and I think the final part of the comment by @michaeltlombardi above is really relevant to you) is that a lot of these problems are already solved by existing higher-order tools, so you may save time by looking into one of those.

For example, if you want to stay in the DSC space, try https://github.com/dsccommunity/DscWorkshop (currently only supports DSC 1.1 and 2.0, but fully functional in those spaces) with Azure Machine Configuration (uses DSC 2.0).

Or Ansible / Chef / Puppet etc.

DSC 3.0 is just a very small part of the IaC infrastructure space and Mike's advice above I think is on point: 'if you don't have to try building your own tooling from the ground up, don't. If anyone in your organization is already using a higher order configuration management tool, start working with them to see whether and how you can collaborate. If those tools don't yet support DSC, starting the conversation and showing a prototype can help you move forward.'

I wouldn't personally advise using DSC 3.0 'bare' in production for any complex medium or large server environment (without an existing higher order tool) until the compliance, logging and orchestration side of things is available via something like Azure Machine Configuration.

[hansze]

Hi, I totally agree using bare DSCv3.x would be best and to integrate with orchestration tools that exist. On that note I was wondering if there are any updates on how discussion are going with those who own Machine Configuration (Azure Policy). Are there talks about including integration of DSCv.3 into their policy packaging (i.e. New-GuestConfigurationPackage). It would be great if the same packaging which could be delivered both with Azure Policy as well as without.