Difficulty in identifying the source of functions used in a project

[ahpooch]

When studying DSC repositories, at first I have encountered difficulties in determining where certain custom functions or cmdlets in the code come from. Can we do something to make this easier for other contributors?

Let’s examine the issue using an example from this repository (DscResource.Base).
In the file source\Classes\010.ResourceBase.ps1, among others, the following cmdlets are used:

• Get-DscProperty
• Test-DscProperty
• Compare-DscParameterState

In some DSC repositories, I’ve seen annotations in .NOTES that help identify where a function or cmdlet is defined, like this:

Used Functions:  
            Name                                | Module  
            ---------------------------------   |--------------------------  
            Get-DscProperty                     | DscResource.Common  
            Test-DscProperty                    | DscResource.Common  
            Compare-DscParameterState           | DscResource.Common  

Initially, I wanted to create a PR modifying 010.ResourceBase.ps1 by adding the changes to the .NOTES section as described above. However, this approach could create unnecessary maintenance overhead. Even if we only document the sources of cmdlets from external modules, these informational blocks would need to be added to every file that uses external cmdlets.

That said, simply ensuring that information about external cmdlets can be found via a global search across the module’s files would already greatly improve the situation.
To achieve this, we could:

• Add a dedicated section in README.md,
• Create a separate file (e.g., ExternalFunctions.md), or
• Include the information in RequiredModules.psd1.

Personally, I think a separate, well-structured file in the repository’s root would be more convenient and useful than the other options.
It could be named like

• DEPENDENCIES.md
• EXTERNAL-FUNCTIONS.md
• EXTERNAL-MODULES.md
• USED-MODULES.md
• MODULES-USED.md
• FUNCTION-REFERENCES.md
• MODULES-REFERENCES.md
  etc.

I’d like to hear the maintainers’ opinions on this issue before submitting a PR on this topic.

[johlju]

Any commands that are used inside the classes comes from a module defined in the build configuration:

DscResource.Base/build.yaml

Lines 71 to 79 in d7c5fbe

	####################################################
	# Dependent Modules Configuration (Sampler) #
	####################################################
	NestedModule:
	DscResource.Common:
	CopyOnly: true
	Path: ./output/RequiredModules/DscResource.Common
	AddToManifest: false
	Exclude: PSGetModuleInfo.xml

The actual module is downloaded when resolving dependencies so that it possible to pin to a specific version (if necessary):

DscResource.Base/RequiredModules.psd1

Lines 10 to 11 in d7c5fbe

	# Build dependencies needed for using the module
	'DscResource.Common' = 'latest'

When the build pipeline runs it copies the module from requiredmodules folder to the built directory.

In the prefix.ps1 (which is baked into the built module's .psm1 file) the built module imports the dependency module:

DscResource.Base/source/prefix.ps1

Lines 1 to 2 in d7c5fbe

	$script:dscResourceCommonModulePath = Join-Path -Path $PSScriptRoot -ChildPath 'Modules/DscResource.Common'
	Import-Module -Name $script:dscResourceCommonModulePath

How that work should probably be documented in the Sampler project since it is those templates and CI/CD pipeline we use, the contributing guidelines could link to such documentation. We should not document this per project as that gives another maintenance overhead that most likely won't be uphold.

[johlju]

If such documentation (private/public function - command used vs dependent module and the required version) should exist then that need to be automated in the build pipeline, and should probably be a task in DscResource.DocGenerator. 🤔 It could possible be added as a separate markdown file in the built wiki content with a front matter category 'Development', the CONTRIBUTING.mnd can link to that. Just an idea, might be better options.

[ahpooch]

@johlju Thank you for the valuable information! I agree that documentation about the Sampler module's principles should reside in the Sampler module itself.

However, my question was slightly different. In any DSC project, it is certainly possible to determine where external modules like DscResource.Base or DscResource.Common are loaded from. But it is much harder to pinpoint the exact source of specific cmdlets. For example, when encountering the use of Get-DscProperty in DscResource.Base, there is no quick way to understand which imported module contains its implementation or documentation.

One possible solution might be to first build the module locally to fetch the external module files, allowing global search to work. However, when studying the project directly on GitHub, there is no way to tell that Get-DscProperty is imported from DscResource.Common and not some other module.

A dedicated file could solve this issue.

For example, we could add a DEPENDENCIES.md file with content structure like this

# External modules used in this repository

| Module | Puprose | Link |
| -- | -- | -- |
| Plaster | Template-based file and project generator. File generation is performed using crafted templates | [PowerShellOrg/Plaster](https://github.com/PowerShellOrg/Plaster) |
| SampleModule | Plaster Template | [gaelcolas/SampleModule](https://github.com/gaelcolas/SampleModule) |
| PlatyPS | External markdown help files and cab files (for Update-Help) generator. | [PowerShell/platyPS](https://github.com/PowerShell/platyPS) |
| PSDepend | PowerShell dependency handler | [RamblingCookieMonster/PSDepend](https://github.com/RamblingCookieMonster/PSDepend) |
| InvokeBuild | Build and test automation tool | [nightroman/Invoke-Build](https://github.com/nightroman/Invoke-Build) |
| PSScriptAnalyzer | Code Quality Assurance | [PowerShell/PSScriptAnalyzer](https://github.com/PowerShell/PSScriptAnalyzer) |
| Pester | Unit and Integration tests | [pester/Pester](https://github.com/pester/Pester) |
| ModuleBuilder | Helps to ship module as one big file while working on it in different small files | [PoshCode/ModuleBuilder](https://github.com/PoshCode/ModuleBuilder) |
| ChangelogManagement | ChangelogManagement is a PowerShell module for reading and manipulating changelog files in [Keep a Changelog 1.0.0](https://keepachangelog.com/en/1.0.0/) format. | [natescherer/ChangelogManagement](https://github.com/natescherer/ChangelogManagement) |
| DscResource.Common | Common functions that are used in DSC resources. | [dsccommunity/DscResource.Common](https://github.com/dsccommunity/DscResource.Common) |

## External CmdLets and functions used in source files

| CmdLet or function | Source |
| -- | -- |
| Get-DscProperty | DscResource.Common |
| Test-DscProperty | DscResource.Common |
| Compare-DscParameterState | DscResource.Common |

It'll make work with project easier for new contributors as information would be present explicitly

External modules used in this repository

Module	Puprose	Link
Plaster	Template-based file and project generator. File generation is performed using crafted templates	PowerShellOrg/Plaster
SampleModule	Plaster Template	gaelcolas/SampleModule
PlatyPS	External markdown help files and cab files (for Update-Help) generator.	PowerShell/platyPS
PSDepend	PowerShell dependency handler	RamblingCookieMonster/PSDepend
InvokeBuild	Build and test automation tool	nightroman/Invoke-Build
PSScriptAnalyzer	Code Quality Assurance	PowerShell/PSScriptAnalyzer
Pester	Unit and Integration tests	pester/Pester
ModuleBuilder	Helps to ship module as one big file while working on it in different small files	PoshCode/ModuleBuilder
ChangelogManagement	ChangelogManagement is a PowerShell module for reading and manipulating changelog files in Keep a Changelog 1.0.0 format.	natescherer/ChangelogManagement
DscResource.Common	Common functions that are used in DSC resources.	dsccommunity/DscResource.Common

External CmdLets and functions used in source files

CmdLet or function	Source
Get-DscProperty	DscResource.Common
Test-DscProperty	DscResource.Common
Compare-DscParameterState	DscResource.Common

[ahpooch]

If such documentation (private/public function - command used vs dependent module and the required version) should exist then that need to be automated in the build pipeline, and should probably be a task in DscResource.DocGenerator. 🤔 It could possible be added as a separate markdown file in the built wiki content with a front matter category 'Development', the CONTRIBUTING.mnd can link to that. Just an idea, might be better options.

@johlju I like the automation idea. However, the effort required to create a concise file describing the modules used in the project and sources of external functions in the source directory is incomparably small.

I believe that for such automation to ever be implemented, we first need to establish the practice of maintaining a manually updated file. This file shouldn't contain excessive information that would require frequent updates. An example of the file's content and suggested naming can be found in my comment above.