[Release/8.0-staging] Reduce net core app current package dependencies (#107161)

* Avoid package dependencies on libraries in the shared framework (#106172)

* Avoid package dependencies on libraries in the shared framework

We can avoid these dependencies since we can count on the library being
part of the shared framework.  Fewer dependencies means less packages
downloaded, less for customers to service, less copied into the output
directory when serviced.

* Add warning code.

* Address feedback

* Add an option to enable servicing for transitive dependencies

* Enable all packages that removed dependencies

* Add additional packages required for up-stack servicing

* Permit settting ServiceTransitiveDependencies in projects as well

Author: ericstj

docs/coding-guidelines/libraries-packaging.md

@@ -18,6 +18,14 @@ Source generators and analyzers can be included in the shared framework by addin
 
 Removing a library from the shared framework is a breaking change and should be avoided.
 
+### References to libraries in the shared framework that produce packages
+
+It's beneficial to avoid project references to libraries that are in the shared framework because it makes the package graph smaller which reduces the number of packages that require servicing and the number of libraries that end up being copied into the application directory.
+
+If a dependency is part of the shared framework a project/package reference is never required on the latest version (`NetCoreAppCurrent`).  A reference is required for previous .NET versions even if the dependency is part of the shared framework if the project you are building targets .NETStandard and references the project there.  You may completely avoid a package dependency on .NETStandard and .NET if it's not needed for .NETStandard (for example - if it is an implementation only dependency and you're building a PNSE assembly for .NETStandard).
+
+Warning NETPKG0001 is emitted when you have an unnecessary reference to a library that is part of the shared framework.  To avoid this warning, make sure your ProjectReference is conditioned so that it doesn't apply on `NetCoreAppCurrent`.
+
 ## Transport package
 
 Transport packages are non-shipping packages that dotnet/runtime produces in order to share binaries with other repositories.

docs/project/library-servicing.md

@@ -16,6 +16,10 @@ If a library is packable (check for the `<IsPackable>true</IsPackable>` property
 
 When you make a change to a library & ship it during the servicing release, the `ServicingVersion` must be bumped. This property is found in the library's source project. It's also possible that the property is not in that file, in which case you'll need to add it to the library's source project and set it to 1. If the property is already present in your library's source project, just increment the servicing version by 1.
 
+## Optionally ensure all up-stack packages are also produced
+
+If you wish to ensure that every package that references a serviced package is also serviced itself, you can enable validation by setting `ServiceTransitiveDependencies` to true.  This can be done in an individual project, or globally.  When doing this then building the repo, eg: `build libs -allConfigurations` you'll see errors from any project that didn't enable servicing.  Reasons for forcing packages which depend on your package to service are security servicing or removing dependencies.
+
 ## Test your changes
 
 All that's left is to ensure that your changes have worked as expected. To do so, execute the following steps:

eng/packaging.targets

@@ -196,6 +196,21 @@
     </ItemGroup>
   </Target>
 
+  <Target Name="WarnOnProjectReferenceToFrameworkAssemblies"
+          BeforeTargets="IncludeTransitiveProjectReferences"
+          Condition="'$(TargetFramework)' == '$(NetCoreAppCurrent)' and
+                     '@(ProjectReference)' != ''">
+    <!-- Find project references that overlap with NetCoreApp, are direct (NuGetPackageId is not set), actually referenced (ReferenceOutputAssembly), and not hidden with PrivateAssets
+         ProjectReferences can opt out of this checking by setting AllowFrameworkPackageReference, though they should not. -->
+    <Warning Text="Project reference '%(ProjectReference.Identity)' is a reference to a framework assembly and is not required in $(NetCoreAppCurrent) (NetCoreAppCurrent)."
+             Code="NETPKG0001"
+             Condition="$(NetCoreAppLibrary.Contains('%(ProjectReference.Filename);')) and
+                        '%(ProjectReference.ReferenceOutputAssembly)' != 'false' and
+                        '%(ProjectReference.NuGetPackageId)' == '' and
+                        '%(ProjectReference.PrivateAssets)' != 'all' and
+                        '%(ProjectReference.AllowFrameworkPackageReference)' != 'true'" />
+  </Target>
+
   <Target Name="GenerateMultiTargetRoslynComponentTargetsFile"
           Inputs="$(MSBuildProjectFullPath);$(_MultiTargetRoslynComponentTargetsTemplate)"
           Outputs="$(MultiTargetRoslynComponentTargetsFileIntermediatePath)">
@@ -303,12 +318,47 @@
     </ItemGroup>
   </Target>
 
-  <Target Name="ValidateServicingVersionIsProperlySet"
+  <ItemDefinitionGroup>
+    <TargetPathWithTargetPlatformMoniker>
+      <!-- When ServiceTransitiveDependencies is set, flow the packaging state -->
+      <GeneratePackageOnBuild Condition="'$(ServiceTransitiveDependencies)' == 'true'">$(GeneratePackageOnBuild)</GeneratePackageOnBuild>
+    </TargetPathWithTargetPlatformMoniker>
+  </ItemDefinitionGroup>
+
+  <!-- Flows the list of ProjectReferences that are enabled for packaging when building a multi-targeting project -->
+  <Target Name="_AddTransitiveServicedPackagesToOutput"
+          AfterTargets="GetTargetPathWithTargetPlatformMoniker"
+          Condition="'$(IsInnerBuild)' == 'true'">
+    <PropertyGroup>
+      <_TransitiveServicedPackages
+          Condition="'%(ReferencePath.ReferenceSourceTarget)' == 'ProjectReference' and
+                     '%(ReferencePath.GeneratePackageOnBuild)' == 'true'"
+                     >@(ReferencePath->'%(OriginalProjectReferenceItemSpec)')</_TransitiveServicedPackages>
+    </PropertyGroup>
+    <ItemGroup>
+      <TargetPathWithTargetPlatformMoniker TransitiveServicedPackages="$(_TransitiveServicedPackages)" />
+    </ItemGroup>
+  </Target>
+
+  <!-- Validate that ServicingVersion is set and packing is enabled.  Runs once in the outer build (or only build if no outer build exists). -->
+  <Target Name="ValidateServicingProperties"
           Condition="'$(PreReleaseVersionLabel)' == 'servicing' and
                      '$(PackageUseIncrementalServicingVersion)' == 'true' and
-                     '$(DotNetBuildFromSource)' != 'true'"
-          AfterTargets="GenerateNuspec">
-    <Error Condition="'$(ServicingVersion)' == '0'" Text="ServicingVersion is set to 0 and it should be an increment of the patch version from the last released package." />
+                     '$(DotNetBuildFromSource)' != 'true' and
+                     '$(IsInnerBuild)' != 'true'"
+          AfterTargets="Build">
+    <ItemGroup>
+      <TransitiveServicedPackages Include="%(InnerOutput.TransitiveServicedPackages)" Condition="'$(IsCrossTargetingBuild)' == 'true'" />
+      <TransitiveServicedPackages Include="'%(ReferencePath.OriginalProjectReferenceItemSpec)"
+                                  Condition="'%(ReferencePath.ReferenceSourceTarget)' == 'ProjectReference' and
+                                             '%(ReferencePath.GeneratePackageOnBuild)' == 'true'" />
+    </ItemGroup>
+    <Error Condition="'$(ServicingVersion)' == '0' and '$(GeneratePackageOnBuild)' == 'true'"
+           Text="ServicingVersion is set to 0 and it should be an increment of the patch version from the last released package." />
+
+    <Error Condition="'$(GeneratePackageOnBuild)' != 'true' and
+                  '@(TransitiveServicedPackages)' != ''"
+        Text="This project did not set GeneratePackageOnBuild, but dependencies '@(TransitiveServicedPackages)' did.  Please ship this project by setting GeneratePackageOnBuild to true and incrementing ServicingVersion." />
   </Target>
 
 </Project>

src/libraries/Microsoft.Extensions.Caching.Memory/src/Microsoft.Extensions.Caching.Memory.csproj

@@ -4,6 +4,8 @@
     <TargetFrameworks>$(NetCoreAppCurrent);$(NetCoreAppPrevious);$(NetCoreAppMinimum);netstandard2.0;$(NetFrameworkMinimum)</TargetFrameworks>
     <EnableDefaultItems>true</EnableDefaultItems>
     <IsPackable>true</IsPackable>
+    <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
+    <ServicingVersion>1</ServicingVersion>
     <PackageDescription>In-memory cache implementation of Microsoft.Extensions.Caching.Memory.IMemoryCache.</PackageDescription>
   </PropertyGroup>
 

src/libraries/Microsoft.Extensions.Configuration.Json/src/Microsoft.Extensions.Configuration.Json.csproj

@@ -5,6 +5,8 @@
     <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
     <EnableDefaultItems>true</EnableDefaultItems>
     <IsPackable>true</IsPackable>
+    <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
+    <ServicingVersion>1</ServicingVersion>
     <PackageDescription>JSON configuration provider implementation for Microsoft.Extensions.Configuration. This package enables you to read your application's settings from a JSON file. You can use JsonConfigurationExtensions.AddJsonFile extension method on IConfigurationBuilder to add the JSON configuration provider to the configuration builder.</PackageDescription>
   </PropertyGroup>
 
@@ -13,9 +15,12 @@
     <ProjectReference Include="$(LibrariesProjectRoot)Microsoft.Extensions.Configuration.Abstractions\src\Microsoft.Extensions.Configuration.Abstractions.csproj" />
     <ProjectReference Include="$(LibrariesProjectRoot)Microsoft.Extensions.Configuration.FileExtensions\src\Microsoft.Extensions.Configuration.FileExtensions.csproj" />
     <ProjectReference Include="$(LibrariesProjectRoot)Microsoft.Extensions.FileProviders.Abstractions\src\Microsoft.Extensions.FileProviders.Abstractions.csproj" />
-    <ProjectReference Include="$(LibrariesProjectRoot)System.Text.Json\src\System.Text.Json.csproj" />
     <Compile Include="$(CommonPath)System\ThrowHelper.cs"
              Link="Common\System\ThrowHelper.cs" />
   </ItemGroup>
 
+  <ItemGroup Condition="'$(TargetFramework)' != '$(NetCoreAppCurrent)'">
+    <ProjectReference Include="$(LibrariesProjectRoot)System.Text.Json\src\System.Text.Json.csproj" />
+  </ItemGroup>
+
 </Project>

src/libraries/Microsoft.Extensions.Configuration.UserSecrets/src/Microsoft.Extensions.Configuration.UserSecrets.csproj

@@ -4,6 +4,8 @@
     <TargetFrameworks>$(NetCoreAppCurrent);$(NetCoreAppPrevious);$(NetCoreAppMinimum);netstandard2.0;$(NetFrameworkMinimum)</TargetFrameworks>
     <EnableDefaultItems>true</EnableDefaultItems>
     <IsPackable>true</IsPackable>
+    <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
+    <ServicingVersion>1</ServicingVersion>
     <PackageDescription>User secrets configuration provider implementation for Microsoft.Extensions.Configuration. User secrets mechanism enables you to override application configuration settings with values stored in the local secrets file. You can use UserSecretsConfigurationExtensions.AddUserSecrets extension method on IConfigurationBuilder to add user secrets provider to the configuration builder.</PackageDescription>
   </PropertyGroup>
 

src/libraries/Microsoft.Extensions.Configuration.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj

@@ -4,6 +4,8 @@
     <TargetFrameworks>$(NetCoreAppCurrent);$(NetCoreAppPrevious);$(NetCoreAppMinimum);netstandard2.0;$(NetFrameworkMinimum)</TargetFrameworks>
     <EnableDefaultItems>true</EnableDefaultItems>
     <IsPackable>true</IsPackable>
+    <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
+    <ServicingVersion>1</ServicingVersion>
     <PackageDescription>XML configuration provider implementation for Microsoft.Extensions.Configuration. This package enables you to read configuration parameters from XML files. You can use XmlConfigurationExtensions.AddXmlFile extension method on IConfigurationBuilder to add XML configuration provider to the configuration builder.</PackageDescription>
   </PropertyGroup>
 
@@ -24,7 +26,7 @@
   <ItemGroup Condition="!$([MSBuild]::IsTargetFrameworkCompatible('$(TargetFramework)', 'net7.0'))">
     <Compile Include="$(CoreLibSharedDir)System\Diagnostics\CodeAnalysis\RequiresDynamicCodeAttribute.cs" />
   </ItemGroup>
-  
+
   <ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETFramework'">
     <Reference Include="System.Security" />
   </ItemGroup>

src/libraries/Microsoft.Extensions.DependencyModel/src/Microsoft.Extensions.DependencyModel.csproj

@@ -3,8 +3,8 @@
     <TargetFrameworks>$(NetCoreAppCurrent);$(NetCoreAppPrevious);$(NetCoreAppMinimum);netstandard2.0;$(NetFrameworkMinimum)</TargetFrameworks>
     <EnableDefaultItems>true</EnableDefaultItems>
     <IsPackable>true</IsPackable>
-    <GeneratePackageOnBuild>false</GeneratePackageOnBuild>
-    <ServicingVersion>1</ServicingVersion>
+    <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
+    <ServicingVersion>2</ServicingVersion>
     <PackageDescription>Provides abstractions for reading `.deps` files. When a .NET application is compiled, the SDK generates a JSON manifest file (`&lt;ApplicationName&gt;.deps.json`) that contains information about application dependencies. You can use `Microsoft.Extensions.DependencyModel` to read information from this manifest at run time. This is useful when you want to dynamically compile code (for example, using Roslyn Emit API) referencing the same dependencies as your main application.
 
 By default, the dependency manifest contains information about the application's target framework and runtime dependencies. Set the PreserveCompilationContext project property to `true` to additionally include information about reference assemblies used during compilation.</PackageDescription>
@@ -22,18 +22,18 @@ By default, the dependency manifest contains information about the application's
   <ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETCoreApp'">
     <Compile Remove="**\*.netstandard.cs" />
   </ItemGroup>
-  
+
   <ItemGroup Condition="'$(TargetFrameworkIdentifier)' != '.NETCoreApp'">
     <Compile Remove="**\*.netcoreapp.cs" />
     <Compile Include="$(CoreLibSharedDir)System\Diagnostics\CodeAnalysis\RequiresAssemblyFilesAttribute.cs" />
     <Compile Include="$(CoreLibSharedDir)System\Diagnostics\CodeAnalysis\UnconditionalSuppressMessageAttribute.cs" />
   </ItemGroup>
 
   <ItemGroup>
-    <InternalsVisibleTo Include="Microsoft.Extensions.DependencyModel.Tests" /> 
+    <InternalsVisibleTo Include="Microsoft.Extensions.DependencyModel.Tests" />
   </ItemGroup>
 
-  <ItemGroup>
+  <ItemGroup Condition="'$(TargetFramework)' != '$(NetCoreAppCurrent)'">
     <ProjectReference Include="$(LibrariesProjectRoot)System.Text.Encodings.Web\src\System.Text.Encodings.Web.csproj" />
     <ProjectReference Include="$(LibrariesProjectRoot)System.Text.Json\src\System.Text.Json.csproj" />
   </ItemGroup>
@@ -46,5 +46,5 @@ By default, the dependency manifest contains information about the application's
   <ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETFramework'">
     <Reference Include="System.Runtime" />
   </ItemGroup>
-  
+
 </Project>

src/libraries/Microsoft.Extensions.Diagnostics.Abstractions/src/Microsoft.Extensions.Diagnostics.Abstractions.csproj

@@ -4,6 +4,8 @@
     <TargetFrameworks>$(NetCoreAppCurrent);$(NetCoreAppPrevious);$(NetCoreAppMinimum);netstandard2.0;$(NetFrameworkMinimum)</TargetFrameworks>
     <EnableDefaultItems>true</EnableDefaultItems>
     <IsPackable>true</IsPackable>
+    <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
+    <ServicingVersion>1</ServicingVersion>
     <!-- Disabling baseline validation since this is a brand new package.
        Once this package has shipped a stable version, the following line
        should be removed in order to re-enable validation. -->
@@ -31,8 +33,11 @@ Microsoft.Extensions.Diagnostics.Metrics.MetricsOptions</PackageDescription>
     <Compile Include="$(CoreLibSharedDir)System\Diagnostics\CodeAnalysis\DynamicallyAccessedMemberTypes.cs" />
   </ItemGroup>
 
-  <ItemGroup>
+  <ItemGroup Condition="'$(TargetFramework)' != '$(NetCoreAppCurrent)'">
     <ProjectReference Include="$(LibrariesProjectRoot)System.Diagnostics.DiagnosticSource\src\System.Diagnostics.DiagnosticSource.csproj" />
+  </ItemGroup>
+
+  <ItemGroup>
     <ProjectReference Include="$(LibrariesProjectRoot)Microsoft.Extensions.DependencyInjection.Abstractions\src\Microsoft.Extensions.DependencyInjection.Abstractions.csproj" />
     <ProjectReference Include="$(LibrariesProjectRoot)Microsoft.Extensions.Options\src\Microsoft.Extensions.Options.csproj" />
   </ItemGroup>

src/libraries/Microsoft.Extensions.Diagnostics/src/Microsoft.Extensions.Diagnostics.csproj

@@ -8,6 +8,8 @@
        should be removed in order to re-enable validation. -->
     <DisablePackageBaselineValidation>true</DisablePackageBaselineValidation>
     <IsPackable>true</IsPackable>
+    <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
+    <ServicingVersion>1</ServicingVersion>
     <PackageDescription>This package includes the default implementation of IMeterFactory and additional extension methods to easily register it with the Dependency Injection framework.</PackageDescription>
   </PropertyGroup>