Merge branch 'main' into UseNETCoreAppTasks

Author: ViktorHofer

.vault-config/dnceng-partners-kv.yaml

@@ -71,6 +71,15 @@ secrets:
         location: EngKeyVault
       gitHubBotAccountName: dotnet-bot
 
+  # repo, workflow scopes (classic token)
+  BotAccount-dotnet-renovate-bot-PAT:
+    type: github-access-token
+    parameters:
+      gitHubBotAccountSecret:
+        name: BotAccount-dotnet-renovate-bot
+        location: EngKeyVault
+      gitHubBotAccountName: dotnet-renovate-bot
+
   roslyn-dn-bot-devdiv-build-r-release-r-code-r:
     type: azure-devops-access-token
     parameters:

.vault-config/product-builds-engkeyvault.yaml

@@ -22,6 +22,11 @@ secrets:
     parameters:
       Name: dotnet-bot
 
+  BotAccount-dotnet-renovate-bot:
+    type: github-account
+    parameters:
+      Name: dotnet-renovate-bot
+
   #Publish-Build-Assets
   BotAccount-dotnet-maestro-bot-PAT:
     type: github-access-token

Documentation/Renovate.md

@@ -0,0 +1,82 @@
+# Renovate Dependency Update Tool
+
+## Introduction
+
+This document outlines the integration of [Renovate](https://github.com/renovatebot/renovate) into Arcade to automate dependency updates.
+Renovate is an automated dependency update tool that generates PRs for updating dependencies from a wide variety of sources.
+
+Renovate is similar to Dependabot in its purpose.
+Dependabot should be used when possible.
+However, Renovate supports a much broader range of [dependency types](https://docs.renovatebot.com/modules/datasource/), most notably Docker and GitHub releases.
+For example, Renovate can automatically generate a PR to update a referenced Docker image when a newer version is available.
+
+### Example Scenarios
+
+Here are two scenarios demonstrating the usefulness of Renovate automatically making dependency updates:
+
+#### Container Tags
+
+Repos that reference container tags from the [dotnet/dotnet-buildtools-prereqs-docker](https://github.com/dotnet/dotnet-buildtools-prereqs-docker) repo need to maintain those tags to ensure they are supported.
+
+This can be as simple as automatically updating to a new major version of Linux distro:
+
+```diff
+-mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64
++mcr.microsoft.com/dotnet-buildtools/prereqs:debian-12-helix-amd64
+```
+
+Or automatically pinning to the latest version of a tag by it's digest value:
+
+```diff
+-mcr.microsoft.com/dotnet-buildtools/prereqs:debian-12-helix-amd64@sha256:b99da50c4cb425e72ee69c2b8c1fdf99e0f71059aee19798e2f9310141ea48fb
++mcr.microsoft.com/dotnet-buildtools/prereqs:debian-12-helix-amd64@sha256:6bb6fef390e6f09a018f385e346b0fe5999d7662acd84ca2655e9a3c3e622b71
+```
+
+Renovate can detect when these new container images are available and submit PRs to update sources accordingly.
+
+Related issue: [Automatically update image references in consuming repos (#1321)](https://github.com/dotnet/dotnet-buildtools-prereqs-docker/issues/1321)
+
+#### GitHub Release
+
+There are many cases where a version of OSS published via GitHub releases is being referenced by a .NET repository.
+Those versions can be kept updated automatically as new releases occur.
+
+For example, there are Dockerfiles in the [dotnet/dotnet-buildtools-prereqs-docker](https://github.com/dotnet/dotnet-buildtools-prereqs-docker) repo which reference the LLVM version that can be maintained by having Renovate automatically check for new [LLVM releases](https://github.com/llvm/llvm-project/releases).
+
+```diff
+-LLVM_VERSION=19.1.7
++LLVM_VERSION=20.1.0
+```
+
+## Design
+
+### Fork Mode
+
+Protecting GitHub repositories in the dotnet organization from direct access by the Renovate tool is crucial.
+Renovate will be used in fork mode, limiting its permissions to forked repositories.
+This avoids giving write permissions to Renovate on any dotnet repository.
+This means that Renovate acts as though it's any other outside contributor, not requiring any special permissions in the dotnet organization or repositories.
+
+A GitHub bot account, `dotnet-renovate-bot`, is used to manage the Renovate operations.
+This account has the ability to create forks from dotnet repositories, which will be the source of the head branch for PRs that are created.
+The PRs generated by Renovate will be done using this bot account.
+Renovate also handles synchronization from the upstream branch automatically.
+GitHub scopes required by this account: `repo`, `workflow`.
+
+### Repo Usage
+
+Arcade provides an Azure DevOps pipeline YAML job template that repositories should utilize when making use of Renovate.
+This template handles the execution of Renovate, ensuring a standardized approach across all repositories.
+Repositories wishing to make use of Renovate can reference this template from a pipeline YAML file, setting the schedule trigger as desired.
+Consuming repositories are responsible for providing their own [Renovate configuration file](https://docs.renovatebot.com/configuration-options/) that describes which dependencies should be updated.
+
+Repositories are in control of how often Renovate will run via the pipeline schedule trigger (once a day, once a week, etc).
+Typically, a pipeline would be configured to update all dependencies at once in which case all dependencies are described in one Renovate config file.
+But in some cases, it may be desired to have certain dependencies updated on a different schedule than others.
+In that case, the repo maintainer would define multiple pipelines with those separate schedules, all configured to run Renovate.
+Importantly, each of those pipelines would need to reference a separate Renovate config file which defines the scope of the updates that will be targeted.
+Lastly, a maintainer is free to manually run the pipeline if an expected update is needed right away.
+
+## Renovate Configuration Patterns
+
+TBD

eng/common/core-templates/job/source-build.yml

@@ -60,7 +60,7 @@ jobs:
       pool:
         ${{ if eq(variables['System.TeamProject'], 'public') }}:
           name: $[replace(replace(eq(contains(coalesce(variables['System.PullRequest.TargetBranch'], variables['Build.SourceBranch'], 'refs/heads/main'), 'release'), 'true'), True, 'NetCore-Svc-Public' ), False, 'NetCore-Public')]
-          demands: ImageOverride -equals build.ubuntu.2204.amd64
+          demands: ImageOverride -equals Azure-Linux-3-Amd64-Public
         ${{ if eq(variables['System.TeamProject'], 'internal') }}:
           name: $[replace(replace(eq(contains(coalesce(variables['System.PullRequest.TargetBranch'], variables['Build.SourceBranch'], 'refs/heads/main'), 'release'), 'true'), True, 'NetCore1ESPool-Svc-Internal'), False, 'NetCore1ESPool-Internal')]
           image: Azure-Linux-3-Amd64
@@ -69,10 +69,10 @@ jobs:
       pool:
         ${{ if eq(variables['System.TeamProject'], 'public') }}:
           name: $[replace(replace(eq(contains(coalesce(variables['System.PullRequest.TargetBranch'], variables['Build.SourceBranch'], 'refs/heads/main'), 'release'), 'true'), True, 'NetCore-Svc-Public' ), False, 'NetCore-Public')]
-          demands: ImageOverride -equals Build.Ubuntu.2204.Amd64.Open
+          demands: ImageOverride -equals Azure-Linux-3-Amd64-Public
         ${{ if eq(variables['System.TeamProject'], 'internal') }}:
           name: $[replace(replace(eq(contains(coalesce(variables['System.PullRequest.TargetBranch'], variables['Build.SourceBranch'], 'refs/heads/main'), 'release'), 'true'), True, 'NetCore1ESPool-Svc-Internal'), False, 'NetCore1ESPool-Internal')]
-          demands: ImageOverride -equals Build.Ubuntu.2204.Amd64
+          demands: ImageOverride -equals Azure-Linux-3-Amd64
   ${{ if ne(parameters.platform.pool, '') }}:
     pool: ${{ parameters.platform.pool }}
 

src/Microsoft.DotNet.Arcade.Sdk/tools/Sign.props

@@ -32,6 +32,7 @@
     <CertificatesSignInfo Include="MacDeveloperVNextHardenWithNotarization" MacCertificate="MacDeveloperVNextHarden" MacNotarizationAppName="dotnet" />
     <!-- Certificates Which support detached signatures -->
     <CertificatesSignInfo Include="LinuxSignTar" SupportsDetachedSignature="true" />
+    <CertificatesSignInfo Include="LinuxSign500180PGP" SupportsDetachedSignature="true" />
   </ItemGroup>
 
   <!-- Only publish packages that contain this build's Target RID in the name.

src/Microsoft.DotNet.Arcade.Sdk/tools/TargetingPacks.BeforeCommonTargets.targets

@@ -1,8 +1,10 @@
 <Project>
 
   <PropertyGroup>
+    <!-- Update downlevel known packs when building from source in the VMR and targeting a non-current .NETCoreApp TFM. -->
     <UpdateDownlevelKnownPacks Condition="'$(UpdateDownlevelKnownPacks)' == '' and
       '$(DotNetBuildSourceOnly)' == 'true' and
+      '$(DotNetBuildFromVMR)' == 'true' and
       '$(TargetFrameworkIdentifier)' == '.NETCoreApp' and
       '$(TargetFrameworkVersion)' != '' and
       $([MSBuild]::VersionLessThan('$(TargetFrameworkVersion)', '$(BundledNETCoreAppTargetFrameworkVersion)'))">true</UpdateDownlevelKnownPacks>
@@ -153,4 +155,4 @@
     </KnownILLinkPack>
   </ItemGroup>
 
-</Project>
+</Project>