Add lifecycle annotations to images pushed to MCR

[MichaelSimons]

A broader set of services are starting to consume image annotations. Specifically Azure container scanning honors the lifecycle annotation to treat EOL images specially. As a result, the images produced with the .NET Docker infrastructure should have these lifecycle annotations added.

Related links:

• OCI Annotations Spec

• Thread & Vulnerability Management guidance (MSFT internal link)

  How can I mark an image in my ACR as end of life or dictate how long it should be scanned and reported on?
  Attach annotations* to the image called 'lifecycle annotations' which requires:
  
  The ACR Metadata Service to be onboarded for your registry (see callout below)
  [ORAS CLI](https://oras.land/docs/installation/) must be installed
  ORAS CLI must be authenticated to your registry [(See How to Authenticate ORAS CLI)](https://learn.microsoft.com/en-us/azure/container-registry/container-registry-oci-artifacts#sign-in-to-a-registry)
  You can use the ORAS CLI to attach lifecycle annotations by running the following command:
  
  Variables:
  
  $IMAGE refers to a fully qualified image name or digest (e.g. myregistry.azurecr.io/myimage:v1 OR myregistry.azurecr.io/myimage@sha256:fdbd1bc674f77e7a3b8129ecac062a9b7c22de74968e8ea3a46f071f61f3e99f)
  Lifecycle.end-of-life.date should be in the expected format of yyyy-mm-dd
  oras attach --artifact-type application/vnd.microsoft.artifact.lifecycle --annotation "vnd.microsoft.artifact.lifecycle.end-of-life.date=2023-05-12T21:15:10.8343521Z" $IMAGE
  

Work Items:

• Add a new command for annotating image digests for EOL #1334
• EOL annotations pipeline #1340
• Command for generation of EOL annotation data file #1358
• Remove force annotate option #1364
• Add command to wait for EOL annotation ingestion #1367
• Push EOL annotations in publish stage #1413
• EOL annotation race condition #1417
• Roll out
  • [dotnet-docker/nightly] Annotate and remove historical images
  • [dotnet-docker/nightly] Enable annotations in pipeline: Enable EOL annotation publishing dotnet-docker#5837
  • [buildtools-prereqs] Annotate and remove historical images
  • [buildtools-prereqs] Enable annotations in pipeline: Enable matrix trimming and EOL annotations dotnet-buildtools-prereqs-docker#1209
  • [dotnet-docker/main] Annotate and remove historical images
  • [dotnet-docker/main] Enable annotations in pipeline
  • [dotnet-framework-docker] Annotate and remove historical images
  • [dotnet-framework-docker] Enable annotations in pipeline: Enable EOL annotations in pipeline microsoft/dotnet-framework-docker#1184
  • [buildtools-prereqs/imagebuilder] Annotate and remove historical images
  • [buildtools-prereqs/imagebuilder] Enable annotations in pipeline: Enable EOL annotations for Image Builder pipeline #1496
• Clean up _ placeholder tags that we pushed to MAR for historical images that were imported from MAR

[lbussell]

[Triage] This needs some design work done before we start work on this. We need a way to figure out which images to tag as EOL and when without scanning every image in our ACR.

[mthalman]

Principals:

• Images that are currently in support will not have EOL annotations. All other images will.
• EOL annotations should be added after we know an image is already EOL. We do not make the assumption that an image will be EOL in the future because we don't know for sure when that will be (e.g. sometimes we unexpectedly skip servicing for a given month).
• These annotations will apply to both images and multi-arch tags (or, more precisely, to manifests and manifest lists).
• Application of annotations should be automated and included as part of the overall image publishing workflow.

Scenarios

• .NET servicing (e.g. 8.0.1 => 8.0.2)
• Rebuilds outside of .NET servicing (such as when base image updates or forced rebuild for package security updates)
• Distro going out of support (such as when we stop publishing a given Alpine version for the rest of the lifetime of a .NET version)
• .NET major version going out of support

Known Good State

First we need to get to a known good state with some initial setup work. This involves setting EOL date annotation for all out-of-support manifests. We can debate how accurate these dates need to be. For example, all 5.0-based tags could be given an EOL date of 5.0's EOL date. Dangling manifests would probably just be set to an arbitrary date. But the need here is for all of these out-of-support manifests to have an EOL date in the past. Any manifest that is not contained in https://github.com/dotnet/versions/tree/main/build-info/docker is considered to be out-of-support.

Scenarios: Servicing and Rebuilds

To account for the scenarios of servicing and rebuilds, the logic can be integrated into the publishing pipeline. This logic would work by first getting the Dockerfile paths for all the images that are being published. It would then look up those paths in the image info file (prior to updating that file as part of the publish process) to see if that Dockerfile path had been previously published. If that Dockerfile path existed in the image info file, grab the image digest associated with it. That digest should be annotated with an EOL of today's date.

This also needs to check the multi-arch tags. For example, a tag like 6.0.28-alpine3.18 is EOL when 6.0.29 is published. That's a multi-arch tag so it's associated with multiple Dockerfiles. For each Dockerfile that's being published, check whether it's associated with any multi-arch tags. For each of those tags, find all associated images. Check whether all of those Dockerfiles are included in this current publishing run. If they are then the multi-arch tag (manifest list) should be EOL-annotated.

Scenarios: Distro/.NET out of support

To account for distro versions or .NET major versions going out of support, we need to account for Dockerfile removal. If a Dockerfile path is removed from the image info file as part of publishing, the digest associated with that file should be EOL-annotated. Similarly as with servicing, multi-arch tags should be accounted for as well. If a multi-arch tag has been removed from the image info file, the digest for it should be EOL-annotated.

[richlander]

This looks solid.

Perhaps we should delete all -preview tags (before 9.0) at the same thing. I don't see value in marking those for EOL (and leaving them).

[MichaelSimons]

Look good @mthalman. I had a few thoughts/questions as I reviewed your proposal:

1. One scenario variant that we see in buildtools-prereqs is that support for an image variant is dropped. To me this is equivalent to a .NET major version going out of support in the official .NET Docker repo.
2. Regarding Servicing and Rebuilds, you mentioned we would gather the EOL data during publishing. When would the adornment of the actual EOL annotations take place?
3. There are likely some design/implementation challenges in regards to dates/timezones and publishing/ingestion occurring across a date boundary.

[mthalman]

When would the adornment of the actual EOL annotations take place?

I was thinking it would either be at the very end of the publishing stage or as a separate post-publishing stage.

[richlander]

We are certain to get the adornment wrong at least once. We need to ensure we can run this stage on its own after quickly altering some data file.