55
661 .  [ Source Build Configuration] ( #source-build-configuration ) 
771 .  [ Setup CI] ( #setup-ci ) 
8- 1 .  [ Incorporate the new repo into the source build dependency
9-    tree] ( #incorporate-the-new-repo-into-the-source-build-dependency-tree ) 
1081 .  [ Source build repos and the VMR] ( #source-build-repos-and-the-vmr ) 
1191 .  [ Validate] ( #validate ) 
12- 1 .  [ Additional resources] ( #additional-resources ) 
1310
1411## Source Build Configuration  
1512
16- Configuring source build involves setting up files in ` eng/ `  that determine the
17- behavior of source build in the repo.
18- 
19- These changes are all needed before source build will work:
20- 
21- *  [ ` eng/DotNetBuild.props ` ] ( #engdotnetbuildprops )  - Basic properties, such as
22-   repo name.
23- *  [ ` eng/SourceBuildPrebuiltBaseline.xml ` ] ( #engsourcebuildprebuiltbaselinexml )  -
24-   List of allowed prebuilts (approval required).
25- *  [ ` eng/Version.Details.xml ` ] ( #engversiondetailsxml )  - Already exists, but
26-   modifications are needed to pull dependencies from upstream [ intermediate
27-   nupkgs] ( ../planning/arcade-powered-source-build/README.md#intermediate-nupkg-outputsinputs ) .
28- 
29- See the following sections for details:
30- 
31- ### ` eng/DotNetBuild.props `  
32- 
33- ``` xml 
34- <Project >
35- 
36-   <PropertyGroup >
37-     <GitHubRepositoryName >this-repo</GitHubRepositoryName >
38-     <SourceBuildManagedOnly >true</SourceBuildManagedOnly >
39-   </PropertyGroup >
40- 
41- </Project >
42- ``` 
43- 
44- This file contains basic configuration used to restore the correct dependencies
45- (managed-only or not) and produce the right name for the repo's [ intermediate
46- nupkg] .
47- 
48- *  ` this-repo `  should be the same as the repo's name on GitHub, without the
49-   GitHub organization name.
50- *  ` SourceBuildManagedOnly `  defaults to false if omitted. ` true `  means that the
51-   repo doesn't produce any RID-specific artifacts like
52-   ` Microsoft.NETCore.App.Runtime.linux-x64 ` , only managed code.
53- 
54- These two properties determine the name of the [ intermediate nupkg] :
55- ` Microsoft.SourceBuild.Intermediate.$(GitHubRepositoryName)[.$(RidSuffix)] ` .
56- 
57- It's possible more configuration will be required for specific repos.
58- ` eng/DotNetBuild.props ` , similar to ` eng/Build.props ` , is a place to add extra
59- MSBuild code that can change the way source build behaves.
60- 
61- ### ` eng/SourceBuildPrebuiltBaseline.xml `  
62- 
63- ``` xml 
64- <!--  When altering this file or making other Source Build related changes, include @dotnet/source-build as a reviewer. --> 
65- <!--  See aka.ms/dotnet/prebuilts for guidance on what pre-builts are and how to eliminate them. --> 
66- 
67- <UsageData >
68-   <IgnorePatterns >
69-     <UsagePattern  IdentityGlob =" Microsoft.SourceBuild.Intermediate.*/*"   />
70-   </IgnorePatterns >
71- </UsageData >
72- ``` 
73- 
74- This defines which prebuilt NuGet packages are allowed to be used during the
75- build. Initially, only the source build intermediate packages are allowed. The
76- ` */* `  glob means "any intermediate package, any version". All other prebuilts
77- require approval from the source build team.
78- 
79- When a PR introduces an unexpected prebuilt, PR validation will fail and let us
80- resolve the source-buildability issue before the PR gets merged.
81- 
82- ### CODEOWNERS  
83- 
84- Add the source build team as the
85- [ CODEOWNER] ( https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners ) 
86- of the source build infrastructure.
87- 
88- ```  text 
89- /eng/SourceBuild* @dotnet/source-build 
90- ``` 
91- 
9213### Trying it out locally  
9314
9415If a repo passes build options through to the common arcade build script, a
@@ -98,97 +19,30 @@ source build can be triggered via the following command.
9819./build.sh -sb
9920``` 
10021
101- If a repo does not pass through build options through to the common arcade build
102- script, then you must specify the ` /p:ArcadeBuildFromSource=true `  property as
103- follows.
104- 
105- ``` bash 
106- ./build.sh -c Release --restore --build --pack /p:ArcadeBuildFromSource=true -bl
107- ``` 
108- 
10922>  Note: [ source build is not supported on
11023 Windows] ( https://github.com/dotnet/source-build/issues/1190 ) , only Linux and
11124macOS.
11225
113- After running the build, source build artifacts will be in
114- ` artifacts/sb ` , and the [ intermediate nupkg]  will be something like
115- ` artifacts/packages/*/Microsoft.SourceBuild.Intermediate.*.nupkg ` .
116- 
117- The MSBuild binlog will be placed somewhere like:
118- ` artifacts/log/Release/Build.binlog ` . However, this "outer" binlog doesn't contain
119- the meat of the build: the "inner" build runs inside an ` Exec `  task. The inner
120- binlog will be written to somewhere like:
121- ` artifacts/sb/src/artifacts/log/Release/source-inner-build.binlog ` .
122- 
12326### Excluding components  
12427
12528It is not always necessary or correct to include all repo components in source
12629build.  Components should be excluded if they are not required for the platform
12730being source-built.  Including them expands the source build graph and may not
12831be possible because of licensing. Examples include tests, Windows components
12932(remember source build currently only supports Linux and macOS), etc. To exlcude
130- these components use the ` DotNetBuildFromSource `  msbuild property to
33+ these components use the ` DotNetBuildSourceOnly `  msbuild property to
13134conditionally exclude.
13235
13336``` code 
134- Condition="'$(DotNetBuildFromSource )' != 'true'" 
37+ Condition="'$(DotNetBuildSourceOnly )' != 'true'" 
13538``` 
13639
13740See the [ Unified Build Controls] ( https://github.com/dotnet/arcade/blob/main/Documentation/UnifiedBuild/Unified-Build-Controls.md ) 
13841for more options on how to exclude components from source build.
13942
140- ### ` eng/Version.Details.xml `  
141- 
142- ``` xml 
143- ...
144-     <Dependency  Name =" Microsoft.NETCore.App.Runtime.win-x64"   Version =" 6.0.0-alpha.1.20468.7" 
145-                 CoherentParentDependency =" Microsoft.NET.Sdk"  >
146-       <Uri >https://github.com/dotnet/runtime</Uri >
147-       <Sha >a820ca1c4f9cb5892331e2624d3999c39161fe2a</Sha >
148-       <SourceBuild  RepoName =" runtime"   />
149-     </Dependency >
150- ...
151-     <Dependency  Name =" Microsoft.SourceBuild.Intermediate.source-build-reference-packages" 
152-                 Version =" 5.0.0-alpha.1.20473.1"  >
153-       <Uri >https://github.com/dotnet/source-build-reference-packages</Uri >
154-       <Sha >def2e2c6dc5064319250e2868a041a3dc07f9579</Sha >
155-       <SourceBuild  RepoName =" source-build-reference-packages" 
156-                    ManagedOnly =" true"   />
157-     </Dependency >
158- ...
159- ``` 
160- 
161- Dependency changes may include adding ` SourceBuild `  to existing dependency
162- elements, and adding a new ` source-build-reference-packages `  element.
163- 
164- ` SourceBuild `  causes the source build targets in the Arcade SDK to download
165- [ intermediate nupkg] s from the upstream repo's official build, rather than using
166- prebuilt binaries to fulfill the dependencies. Note that ` RepoName `  is used to
167- calculate the ID of the [ intermediate
168- nupkg] ( ../planning/arcade-powered-source-build/README.md#intermediate-nupkg-outputsinputs ) :
169- the ` Dependency `  ` Name `  is ignored by source build.
170- 
171- Building with the source-built versions of your dependencies also means that any
172- upstream repos will have been built in a source build context, including things
173- projects that are excluded from the source build.  This can help you find issues
174- where your source build depends on an upstream component that isn't actually
175- built in source build.
176- 
177- ` ManagedOnly `  determines whether a RID suffix is necessary on the [ intermediate
178- nupkg] ( ../planning/arcade-powered-source-build/README.md#intermediate-nupkg-outputsinputs ) 
179- ID. For example, running source build on ` dotnet/installer `  with ` linux-x64 ` 
180- with the above example configuration will restore:
181- 
182- *  ` Microsoft.SourceBuild.Intermediate.runtime.linux-x64 ` 
183-   *  ` .linux-x64 `  because ` ManagedOnly `  is not ` true ` .
184- *  ` Microsoft.SourceBuild.Intermediate.source-build-reference-packages ` 
185-   *  Ends with the ` RepoName `  without a suffix because ` ManagedOnly="true" ` .
186- 
18743## Setup CI  
18844
189- Source build needs to run during official builds to create source build
190- [ intermediate nupkg] s for the downstream repos that will consume them. Source
191- build should also run in PR validation builds, to prevent regressions.
45+ Source build should run in PR validation builds, to prevent regressions.
19246
19347Source build CI can be activated with a single flag in the ordinary Arcade SDK
19448jobs template for easy consumption. If a repo can't simply use the Arcade SDK
@@ -297,46 +151,19 @@ the most granular template, and may be useful if some repo-specific preamble or
297151cleanup steps are required, or if the repo already has job matrix templates and 
298152this just happens to fit in nicely. 
299153
300- # ## Official build publishing
301- 
302- Publishing [intermediate nupkg]s in the official build is handled by the 
303- standard Arcade publish infrastructure, which should already be set up in the 
304- repo. The source build steps handle uploading the [intermediate nupkg] to the 
305- pipeline in the standard way that Arcade will detect and publish. 
306- 
307- [intermediate nupkg] :
308-     https://github.com/dotnet/source-build/blob/main/Documentation/planning/arcade-powered-source-build/intermediate-nupkg.md 
309- 
310- # # Incorporate the new repo into the source build dependency tree
311- 
312- Once your repo can be source-built, it is time to register it into the source 
313- build dependency tree. The graph of the product is defined by the 
314- ` eng/Version.Details.xml`   files. This dependency graph starts at 
315- [dotnet/sdk](https://github.com/dotnet/sdk/blob/main/eng/Version.Details.xml). 
316- The dependendecies of repos declared in these files are walked and the first 
317- copy/commit of each repo found in the dependency graph is used. 
318- 
319- Therefore, when adding new repositories, a dependency must be created within the 
320- ` eng/Version.Details.xml`   graph. To do this, **go to the repo which depends on 
321- the new repo and add a [new source build dependency](#engversiondetailsxml) to 
322- the new source build repo**. 
323- 
324154# # Source build repos and the VMR
325155
326156Another effect of adding a new source build repository is that its sources will 
327157be synchronized into the [Virtual Monolithic Repository of 
328158.NET](https://github.com/dotnet/dotnet). The VMR is then where the official 
329- source build happens from. The sources are synchronized once the associated 
330- commit/package flows into `dotnet/sdk`. 
159+ source build happens from. 
331160
332161In order for the sources of the new repo to by synchronized into the VMR, the 
333162repo needs to be registered in the [`source-mappings.json` 
334163file](https://github.com/dotnet/dotnet/blob/main/src/source-mappings.json) which 
335164tells the tooling where from and which sources should be synchronized. Please 
336- open a PR in [`dotnet/sdk`](https://github.com/dotnet/sdk) and add 
337- your repository into `src/VirtualMonoRepo/source-mappings.json`. The name must 
338- match the name declared in the `SourceBuild` tag in `Version.Details.xml` 
339- created in the previous step. 
165+ open a PR in [`dotnet/dotnet`](https://github.com/dotnet/dotnet) and add 
166+ your repository into `src/source-mappings.json`. 
340167
341168# ## Cloaking (filtering) the repository sources
342169
@@ -353,19 +180,7 @@ is needed for the source-built .NET scenario.
353180
354181# # Validate
355182
356- Once the downstream dependency(s) are added to the new repo and those changes 
357- flow into `dotnet/sdk`, a complete .NET product can be built from source. 
358- The repository will be synchronized into the VMR during the first build and the 
359- VMR will be built. This will validate that no prebuilts were added to the system 
360- and everything is functioning correctly. Please notify 
361- [@source-build](https://github.com/orgs/dotnet/teams/source-build) 
362- to be on the lookout for the new repo and they will validate as necessary. 
363- 
364- # # Additional resources
365- 
366- * For more details about how the build executes, see [Arcade's build 
367-   tools](https://github.com/dotnet/arcade/tree/main/src/Microsoft.DotNet.Arcade.Sdk/tools/SourceBuild). 
368- * The source code for the build tasks that run for prebuilt validation and 
369-   intermediate nupkg dependency reading are maintained in 
370-   [Arcade](https://github.com/dotnet/arcade/tree/main/src/Microsoft.DotNet.SourceBuild) 
371-   as well. 
183+ Once the code flows to the VMR, the PR validation legs will ensure that no 
184+ [prebuilts](https://github.com/dotnet/source-build/blob/main/Documentation/eliminating-pre-builts.md#what-is-a-prebuilt) 
185+ were added to the system and everything is functioning correctly. 
186+ If you need help on addressing any prebuilds, reach out to @source-build. 
0 commit comments