[Main] Publish Artifacts step failed after rerun

[Winniexu01]

Related to #4537 and dotnet/sdk@e5a8c45, it seems that @ViktorHofer added the condition succeededOrFailed() to the publish artifacts step and it does not support overwriting artifacts.

Failing build (internal Microsoft link):

##[error]Artifact CentOSStream9_Mono_Offline_CurrentSourceBuiltSdk_x64_Artifacts already exists for build 2620900.

Does below solution make sense to you @MichaelSimons @ellahathaway @ViktorHofer ?

• Add another Publish Artifacts step that only publish on success, and modify the current Publish Artifacts step to include $(System.JobAttempt) that only publish on fail. This will not affect the other pipelines download these artifacts.

[ViktorHofer]

Yes, I had to add the condition back as it was impossible to debug CI issues without artifacts getting uploaded.

[MichaelSimons]

What about appending _Attempt$(System.JobAttempt) to the artifact name? That would make the artifacts unique per run/retry.

[ViktorHofer]

@ellahathaway touched on this in the original issue:

Modify the Publish Artifacts step to include the Job Attempt number. This would mean each run would have it's artifacts uploaded to a distinct artifact name. The downside to this, is there are some dependent jobs (e.g. the source-build sdk content tests) which download the artifacts. They would need to account for the possibility of multiple artifacts existing.

[mthalman]

I've chatted with AzDO folks on this scenario in the past. Their recommended pattern is to use the _Attempt$(System.JobAttempt) naming for the intermediate publishing step. And then have a final step at the end which publishes the same files to a statically name artifact. You're essentially publishing the files twice but they have de-dupe logic that prevents it from doubling the storage size. This way you get the benefits of early access to the artifacts in case of failure but also a known artifact name on a successful build.

[mthalman]

One caveat is that this recommendation was given before all the 1ES-injected tasks like SBOM generation. So I don't know what impact that will have on things as there is a fair amount of overhead in publishing any artifacts at all.