-
Notifications
You must be signed in to change notification settings - Fork 52
(GH-1152, GH-1153) Define cargo workspace and reorganize projects #1161
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
SteveL-MSFT
merged 15 commits into
PowerShell:main
from
michaeltlombardi:gh-1152/main/rust-workspace
Oct 8, 2025
Merged
(GH-1152, GH-1153) Define cargo workspace and reorganize projects #1161
SteveL-MSFT
merged 15 commits into
PowerShell:main
from
michaeltlombardi:gh-1152/main/rust-workspace
Oct 8, 2025
+3,605
−13,840
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
099fbd5
to
31d1402
Compare
Prior to this change, the project didn't use a cargo workspace. Instead, every crate independently managed its dependencies and we associated crates with Rust Analyzer in VS Code through the configuration settings. This change: - Defines a virtual workspace at the repository root, including every crate as a workspace member, defining the release profile, and defining every crate dependency. - Updates the cargo manifest for every crate to inherit their dependencies from the workspace. - Replaces the per-crate cargo lock files with the workspace-level lock file. Now dependency management happens with a single manifest and lock file. - Makes minor changes to the build script to ensure functionality continues as previously defined. Note that this change on its own doesn't drastically improve build times. For build times to improve substantially, we need to: 1. Restructure the build to compile all crates with a single command. When we use `cargo build` at the repository root or the `--workspace` flag, cargo builds every crate and its dependencies together. Cargo uses the dependency graph to only compile each dependency crate once during the build, instead of recompiling every dependency when we compile on a crate-by-crate basis. 1. Enable the use of caching, like with `sccache`. When we enable cached builds, only crates that have been modified since the last build will be recompiled. This _substantially_ improves build times for local development and when we invoke tests after initial compilation. However, both of these changes require more substantial modifications to the build script, so they aren't addressed in this change. Those changes will be addressed in a future commit.
This change moves the library crates, which don't have an executable, into the `lib` folder. This makes discovering shared libraries easier, instead of relying on the `_lib` convention the crates use. This change doesn't rename the libraries, it only moves them. This change doesn't require modifying any code or cargo manifests, except for pointing to the correct path in the virtual workspace.
This change moves the resource subprojects, which have an executable, into the `resources` folder. This makes discovering which projects provide DSC resources easier, instead of checking whether the folder contains any resource manifests. This change doesn't rename the folders, it only moves them. This change doesn't require modifying any code or cargo manifests, except for pointing to the correct path in the virtual workspace and build script.
This change moves the adapter subprojects, which have an executable, into the `adapters` folder. This makes discovering which projects provide DSC resource adapters easier, instead of relying on the `-adapter` suffix convention. This change does rename the folders, dropping the suffix. The adapters aren't rust crates, so this doesn't affect code compilation. This change doesn't require modifying any code or cargo manifests, except for pointing to the correct path in the build script.
This change moves the tree-sitter subprojects, which define grammars, into the `grammars` folder. This makes discovering which projects provide DSC grammars easier, instead of relying on the `tree-sitter-` prefix convention. This change does not rename the folders, it only moves them. This change doesn't require modifying any code or cargo manifests, except for pointing to the correct path in the build script and virtual workspace manifest. This change also updates the `.gitignore` to generically ignore generated grammar files in the `grammars` folder.
This change renames `dsc_lib` to `dsc-lib`. This is done for a few reasons: 1. The general naming convention for rust crates is `kebab-case` for crates, `snake_case` for modules. This aligns the naming convention. Note that kebab-case crate names are still referenced with underscores as `snake_case`, so no code needed to be changed in crates, only the cargo manifests. 1. When we eventually publish the library crates, we will need to use a coherent naming convention. Standardizing on `dsc-lib` as a prefix ensures that the other library crates can follow this convention and more readily be discovered and collected. The other libraries are only intended for use with DSC. This change ensures that the manifests and build script are updated to use the renamed crate. I decided to keep a 1:1 map of crate name and folder, to limit confusion over having more than one `dsc` folder in the project.
Prior to this change, the discovery tests referenced the path to test files for the PowerShell adapter by folder. This change corrects those paths to their new location.
This change updates the WMI adapter test to point to the correct location for the test configuration after the folder move and rename.
… for WinPS Prior to this change, the adapter munged the PSModulePath by removing any path items containing the segment `\powershell\` to ensure no PowerShell 7 modules are used for discovery when invoked with the Windows PowerShell adapter. This implementation was highlighted as a bug after moving and renaming the `powershell-adapter` folder to `adapters/powershell`. The tests showed that none of the test modules in the `Tests` subfolder were discovered by the adapter. The prior implementation removed not only the actual PowerShell 7 module paths, but _any_ module path with `powershell` anywhere in the ancestor folder path. Not only does this cause the tests to fail, but it also prevents users from discovering any modules they might have added in custom locations like `~\dev\powershell` or `D:\Tools\PowerShell`. This change updates the implementation to specifically search for and remove the default module paths for PowerShell _if_ PowerShell is installed. It no longer uses a simple match for any path that contains `powershell` as an ancester folder.
This change renames the `osinfo_lib` crate to `dsc-lib-osinfo`. This is to support eventual publishing of the crates. This crate is a dependency not only for the `osinfo` resource but also the `dsc-lib` crate. When we implement the Rust RDK, we will need to publish the library crates, and this ensures the naming is both consistent and easily discoverable. This change makes the requisite updates to cargo manifests, Rust code, and the build script.
This change renames the `pal` crate to `dsc-lib-pal`. This is to support eventual publishing of the crates. When we implement the Rust RDK, we will need to publish the library crates, and this ensures the naming is both consistent and easily discoverable. This change makes the requisite updates to cargo manifests, Rust code, and the build script.
This change renames the `registry_lib` crate to `dsc-lib-registry`. This is to support eventual publishing of the crates. This crate is a dependency not only for the `registry` resource but also the `sshdconfig` resource. When we implement the Rust RDK, we will need to publish the library crates, and this ensures the naming is both consistent and easily discoverable. This change makes the requisite updates to cargo manifests, Rust code, and the build script.
…y_context` This change renames the `security_context_lib` crate to `dsc-lib-security_context`. This is to support eventual publishing of the crates. This crate is a dependency not only for the `registry` resource but also the `dsc-lib` crate. When we implement the Rust RDK, we will need to publish the library crates, and this ensures the naming is both consistent and easily discoverable. This change makes the requisite updates to cargo manifests, Rust code, and the build script.
Prior to this change, the Rust crates for specific DSC resources used the binary name as the crate name. This causes a few problems: - When using `cargo` commands with the `--package`/`-p` option to limit which workspace packages to use for an operation, some of the packages, like `registry`, conflict with dependencies of the same name. - If we eventually publish any of these crates, they are not obviously related to DSC and might be mistaken as general purpose tools, which we don't currently plan to support. - It isn't obvious in the cargo lock file whether an item is for a workspace package or an external package. This change: 1. Renames the resource crates to `dsc-resource-<name>`. 1. Defines an entry in the `bin` table for the cargo manifest to ensure that the binary name remains the same as it was before the change. We _may_ want to consider renaming the binaries in a future, potentially breaking change, so it is always obvious when a resource is builtin / which binary to use. This change has no effect on the build scripts or workspace manifest, since we explicitly point to folder paths instead of using crate names.
7a1584d
to
00a08cc
Compare
This change adds a reworked build script to the project and a GHA workflow that uses it. Currently this duplicates both the build script and the CI jobs. For now, this enables us to ensure continuity of existing behavior while we iteratively improve the new build script and CI configuration to speed up builds and improve maintainability. The new build script generally provides equivalent functionality with the existing legacy script to the best of my ability. In the process, this change also: - Defines a helper module for build-and-maintenance related functions. This made the build script itself somewhat easier to reason about, and the functions in the module can be used outside of the script. - Moves build data out of the script and into a separate datafile. For now, this puts build data at the root, instead of per-project. In the future, I would like to refactor things to retrieve build data from the individual projects instead. Finally, this change adds a benchmarking comparison script to enable us to programmatically compare the build scripts for local and CI workflows.
00a08cc
to
d2afbc5
Compare
Final builds passing, here is comparison data: Linux:
Comparison:
Test:
DifferenceMultiplier: 1.55x
DifferenceDuration: 2m 22s
Build:
DifferenceDMultiplier: 9.7x
DifferenceDuration: 0m -7s
Total:
DifferenceMultiplier: 1.48x
DifferenceDuration: 3m 45s
New:
Prereq: 1m 31s
Tests: 4m 19s
Build: 1m 58s
Total: 7m 53s
Legacy:
Tests: 6m 41s
Build: 3m 22s
Total: 11m 38s
Windows:
Comparison:
Test:
DifferenceMultiplier: 1.15x
DifferenceDuration: 5m 28s
Build:
DifferenceDMultiplier: 1.19x
DifferenceDuration: 1m 37s
Total:
DifferenceMultiplier: 1.16x
DifferenceDuration: 7m 8s
New:
Prereq: 3m 26s
Tests: 35m 23s
Build: 5m 6s
Total: 44m 8s
Legacy:
Tests: 40m 51s
Build: 10m 9s
Total: 51m 16s
macOS:
Comparison:
Test:
DifferenceMultiplier: 1.87x
DifferenceDuration: 4m 24s
Build:
DifferenceDMultiplier: 1.1x
DifferenceDuration: 0m 3s
Total:
DifferenceMultiplier: 1.46x
DifferenceDuration: 4m 25s
New:
Prereq: 1m 43s
Tests: 5m 3s
Build: 2m 34s
Total: 9m 30s
Legacy:
Tests: 9m 27s
Build: 4m 20s
Total: 13m 55s |
SteveL-MSFT
approved these changes
Oct 8, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Summary
Prior to this change, the project didn't use a cargo workspace. Instead, every crate independently managed its dependencies and we associated crates with Rust Analyzer in VS Code through the configuration settings.
This change:
Note that this change on its own doesn't drastically improve build times. For build times to improve substantially, we need to:
cargo build
at the repository root or the--workspace
flag, cargo builds every crate and its dependencies together. Cargo uses the dependency graph to only compile each dependency crate once during the build, instead of recompiling every dependency when we compile on a crate-by-crate basis.sccache
. When we enable cached builds, only crates that have been modified since the last build will be recompiled. This substantially improves build times for local development and when we invoke tests after initial compilation.However, both of these changes require more substantial modifications to the build script.
To do:
Other changes required as part of this PR:
PSModulePaths
instead of blanket removing any module path withpowershell
as an ancestor folder.PR Context