Skip to content

Commit 5662042

Browse files
MichaelSimonsmthalman
MichaelSimons
and
mthalman
authored
Add package dependency flow documentation (#4123)
* Add package dependency flow documentation * Apply suggestions from code review Co-authored-by: Matt Thalman <[email protected]> * Link to poison doc * Misc formatting edits * wording edits * Updates for incoming isolated package flow work * Wording fixes --------- Co-authored-by: Matt Thalman <[email protected]>
1 parent 77ea15f commit 5662042

File tree

1 file changed

+161
-0
lines changed

1 file changed

+161
-0
lines changed

‎Documentation/package-dependency-flow.md‎

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
# Package Dependency Flow
2+
3+
This document describes how package dependencies are handled within source build.
4+
It describes the mechanisms that exist to control which package versions are used.
5+
6+
## Origins of Packages
7+
8+
A source build must be self-contained, meaning the entire product must be built
9+
from source in an offline environment. To achieve this, all packages dependencies must
10+
be satisfied by one of the following:
11+
12+
### Source-Build-Reference-Packages
13+
14+
The first repo that is built as part of source build is
15+
[source-build-reference-packages](https://github.com/dotnet/source-build-reference-packages).
16+
This repo contains all of the reference packages used to build the product. The repo
17+
contains [tooling to generate new reference packages](https://github.com/dotnet/source-build-reference-packages?tab=readme-ov-file#adding-new-packages).
18+
19+
### Current Source Built Packages
20+
21+
This refers to all of the packages produced in the current build. The set of packages
22+
available to each repo varies based on its build order. For example the msbuild repo
23+
can take a dependency on the current version of Microsoft.CodeAnalysis from roslyn
24+
because roslyn builds before msbuild. Conversely, since roslyn builds before msbuild,
25+
roslyn cannot take a dependency on the current version of Microsoft.Build; it can only
26+
take a dependency on a previously released version.
27+
28+
### Previous Source Built Packages
29+
30+
Because the .NET product uses itself to build, the .NET source build product must be
31+
[bootstrapped](./bootstrapping-guidelines.md). This process allows the packages from the
32+
previous source build release to be used to build the next version of the product. This
33+
provides a means for breaking the product's circular dependencies. For example repos like
34+
[arcade](https://github.com/dotnet/arcade) can self-reference its previous version to
35+
produce the next version.
36+
37+
When referencing previous source built packages, it is important to not leak these
38+
previously built packages into the resulting packages/product. This is considered a
39+
[poison leak](./leak-detection.md) and is not permitted during a source build as it
40+
breaks the notion of building the product entirely from source. This hinders the
41+
ability to service the product.
42+
43+
## Package Versions
44+
45+
Package dependencies that defined using
46+
[Arcade's Darc patterns](https://github.com/dotnet/arcade/blob/main/Documentation/Darc.md)
47+
will get lifted dynamically during a source build if the following conditions are met:
48+
49+
1. The dependency is declared in the Version.Details.xml file.
50+
51+
**Version.Details.xml**
52+
53+
```xml
54+
...
55+
<Dependency Name="System.CommandLine" Version="2.0.0-beta4.24068.1">
56+
<Uri>https://github.com/dotnet/command-line-api</Uri>
57+
<Sha>02fe27cd6a9b001c8feb7938e6ef4b3799745759</Sha>
58+
</Dependency>
59+
...
60+
```
61+
62+
1. A corresponding version property is defined in the Versions.props.
63+
64+
**Versions.props**
65+
66+
```xml
67+
...
68+
<SystemCommandLineVersion>2.0.0-beta4.24068.1</SystemCommandLineVersion>
69+
...
70+
```
71+
72+
1. A repository reference is defined in the
73+
[VMR's project dependency graph](https://github.com/dotnet/dotnet/tree/main/repo-projects).
74+
This reference does not have to be direct, it can be transitive.
75+
76+
**{VMR repo project}.proj**
77+
78+
```xml
79+
...
80+
<RepositoryReference Include="command-line-api" />
81+
...
82+
```
83+
84+
When these conditions are met during a source build, the infrastructure will scan
85+
the Version.Details.xml file and dynamically create two new Versions.props files
86+
containing updated version properties for all non-pinned dependencies.
87+
88+
**PackageVersions.Previous.props:** This will contain version properties with the
89+
package versions from the [previous release of source build](#previous-source-built-packages).
90+
If a new package exists that has never been released before, it will not have a
91+
version property defined.
92+
93+
```xml
94+
...
95+
<SystemCommandLineVersion>2.0.0-beta3</SystemCommandLineVersion>
96+
...
97+
```
98+
99+
**PackageVersions.Current.props:** This will contain version properties with the
100+
package versions from the [current source build](#current-source-built-packages).
101+
If a package comes from a repo that has not been built yet, it will not have a version
102+
property defined.
103+
104+
```xml
105+
...
106+
<SystemCommandLineVersion>2.0.0-beta4</SystemCommandLineVersion>
107+
...
108+
```
109+
110+
These two version.props files get imported by the arcade source build infrastructure after
111+
the repo's Version.props file. Therefore the repo's Versions.props property versions
112+
get overridden by the source build versions. In the case of the `SystemCommandLineVersion`
113+
example, the current source build version, 2.0.0-beta4, would win. All msbuild references
114+
(e.g. project PackageReferences) to these Versions.props properties pick up the newer
115+
versions. This is known as package version lifting since it lifts the originally defined
116+
package version to the current source built version. This behavior only applies to source
117+
build in the context of the [VMR](https://github.com/dotnet/dotnet) (see also
118+
[Repo Level Source Builds](#repo-level-source-builds)).
119+
120+
### Transitive Version Properties
121+
122+
Transitive version properties in your Versions.props file may not work as intended with
123+
source build.
124+
125+
**Versions.props**
126+
127+
```xml
128+
...
129+
<MicrosoftBuildFrameworkVersion>17.7.0-preview-23217-02</MicrosoftBuildFrameworkPackageVersion>
130+
<MicrosoftBuildVersion>$(MicrosoftBuildFrameworkVersion)</MicrosoftBuildPackageVersion>
131+
...
132+
```
133+
134+
**Version.Details.xml**
135+
136+
```xml
137+
...
138+
<Dependency Name="Microsoft.Build.Framework" Version="17.7.0-preview-23217-02">
139+
<Uri>https://github.com/dotnet/msbuild</Uri>
140+
<Sha>2cbc8b6aef648cf21c6a68a0dab7fe09a614e475</Sha>
141+
</Dependency>
142+
<!-- No dependency is declared for "Microsoft.Build". -->
143+
...
144+
```
145+
146+
In this case source build will override the `MicrosoftBuildFrameworkVersion` to the
147+
latest version but the `MicrosoftBuildVersion` will remain set to `17.7.0-preview-23217-02`
148+
because of the property evaluation order. If the desired behavior is for
149+
`MicrosoftBuildVersion` to be set to the same value as `MicrosoftBuildFrameworkVersion`
150+
for source build, then you either need to declare the Microsoft.Build dependency
151+
in the Version.Details.xml file or move the `MicrosoftBuildVersion` assignment outside
152+
of the Versions.props file.
153+
154+
155+
### Repo Level Source Builds
156+
157+
The source build package lifting mechanism is not applicable when building individual
158+
repos in source build mode because it doesn't have the context of the other product
159+
repos or previous source build release. In repo source build mode, the versions of the
160+
packages declared in the Versions.props are used (see also
161+
[backlog issue](https://github.com/dotnet/source-build/issues/3562)).

0 commit comments

Comments
 (0)