The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
2789118	dotnet-roslyn	Test	Log
2789076	dotnet-roslyn	Test	Log
1142485	dotnet/runtime	Send to Helix	Log
1142525	dotnet/runtime	Send to Helix	Log	dotnet/runtime#119472
1142430	dotnet/runtime	Send to Helix	Log
1142439	dotnet/runtime	Send to Helix	Log	dotnet/runtime#119360
1142522	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#63576
1142464	dotnet/runtime	Send to Helix	Log	dotnet/runtime#119396
1142488	dotnet/roslyn	Test	Log
2788896	dotnet-aspnetcore	Run build.cmd helix target	Log	#52827
2788867	dotnet-runtime	Component Detection	Log
2788893	dotnet-dotnet	🛡 Guardian: CredScan (Source)	Log
1142392	dotnet/runtime	Send to Helix	Log	dotnet/runtime#119307
1142331	dotnet/runtime	Publish Build Assets	Log
1142161	dotnet/runtime	Send to Helix	Log
1142242	dotnet/performance	Send job to Helix (Unix)	Log
1142155	dotnet/runtime	Send tests to Helix (Unix)	Log
1142156	dotnet/runtime	Send to Helix	Log
1142159	dotnet/runtime	Send to Helix	Log
1141888	dotnet/runtime	Send to Helix	Log
2788480	dotnet-dotnet	Build (azurelinuxArmCrossContainer)	Log
1141840	dotnet/runtime	Send to Helix	Log
1138051	dotnet/aspnetcore	Run E2E tests	Log	dotnet/aspnetcore#63507
1141557	dotnet/aspire	Run helix tests	Log
1141521	dotnet/runtime	Send to Helix	Log
2788013	dotnet-dotnet	Build (azurelinuxArm64MuslCrossContainer)	Log
1141243	dotnet/runtime	Send tests to Helix (Unix)	Log
2787807	dotnet-install-scripts	Run Tests	Log
1141257	dotnet/runtime	Build product	Log	dotnet/runtime#119432
1141258	dotnet/runtime	Build	Log	dotnet/runtime#119432
2787432	dotnet-dotnet	Build (azurelinuxX64CrossContainer)	Log
1140683	dotnet/runtime	Send tests to Helix (Unix)	Log
1141002	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#79929
2787324	dotnet-sdk	Prepare dotnet/msbuild for formatting	Log	#52957
2787316	dotnet-workload-versions	🛡 Component Governance	Log
1140777	dotnet/runtime	Send to Helix	Log	dotnet/runtime#119413
1140820	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#50445
1140724	dotnet/runtime	Send to Helix	Log	dotnet/runtime#119409
1140788	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#50663
1140696	dotnet/runtime	Send to Helix	Log
1140785	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#50164
1140736	dotnet/runtime	Send to Helix	Log
2787275	dotnet-dotnet	🛡 Guardian: PoliCheck	Log
1140717	dotnet/runtime	Send to Helix (Windows)	Log	dotnet/runtime#119376
1140759	dotnet/performance	Send job to Helix (Windows)	Log	dotnet/performance#4948
1140709	dotnet/runtime	Send to Helix	Log	dotnet/runtime#119408
2787257	dotnet-dotnet	Build (azurelinuxX64CrossContainer)	Log
2782746	dotnet-dotnet	Install MicroBuild plugin	Log
2787073	dotnet-runtime	Run performance job script	Log
2787014	dotnet-windowsdesktop	Build	Log
1140241	dotnet/runtime	Send to Helix	Log
2786981	dotnet-sdk	Run Tests in Helix	Log	#52915
2786926	dotnet-performance	Send job to Helix (Unix)	Log
1140242	dotnet/runtime	Send to Helix	Log
2786904	dotnet-runtime	Send job to Helix (Unix)	Log
2786892	dotnet-aspnetcore	Run build.cmd helix target	Log	#52993
1140137	dotnet/runtime	Send to Helix	Log	dotnet/runtime#119255
2786795	dotnet-runtime	Send tests to Helix (Unix)	Log
2786717	dotnet-dotnet	Build (centOSStreamContainer)	Log
1139914	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#119376
2786665	dotnet-sdk	Run AoT Tests in Helix	Log	#52915
2786653	dotnet-dotnet-buildtools-prereqs-docker	🟪 Build Images	Log
2786666	dotnet-sdk	Run Tests in Helix	Log	#52914
1139884	dotnet/runtime	Send to Helix (Windows)	Log	dotnet/runtime#119369
1139881	dotnet/runtime	Send to Helix (Windows)	Log	dotnet/runtime#118371
1139725	dotnet/runtime	Send to Helix	Log
2786638	dotnet-aspnetcore	Run build.cmd helix target	Log
1139869	dotnet/runtime	Send to Helix	Log	dotnet/runtime#119375
2786616	dotnet-workload-versions	🟣 Build solution	Log
1139806	dotnet/runtime	Send to Helix (Windows)	Log	dotnet/runtime#119355
1139856	dotnet/performance	Send job to Helix (Windows)	Log	dotnet/performance#4939
1139834	dotnet/runtime	Send to Helix	Log	dotnet/runtime#114969
1139809	dotnet/runtime	Send to Helix	Log	dotnet/runtime#119355
2786608	dotnet-efcore	Send job to helix	Log
1139810	dotnet/runtime	Send to Helix	Log	dotnet/runtime#119372
1139797	dotnet/runtime	Send to Helix	Log	dotnet/runtime#119050
1139813	dotnet/runtime	Send to Helix	Log	dotnet/runtime#119373
1139765	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#115502
1139850	dotnet/runtime	Checkout dotnet/runtime@refs/pull/118257/merge to s	Log	dotnet/runtime#118257
1139851	dotnet/runtime	Checkout dotnet/runtime@refs/pull/118257/merge to s	Log	dotnet/runtime#118257
1139742	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#119343
1139758	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#119368
1139751	dotnet/runtime	Send to Helix	Log	dotnet/runtime#119288
1139821	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#63511
1139820	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#63511
1139724	dotnet/runtime	Send to Helix	Log
1139727	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#119324
2786590	dotnet-workload-versions	🟣 Build solution	Log
1139787	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#63261
1139784	dotnet/aspnetcore	Run build.cmd helix target	Log
2786519	dotnet-aspnetcore	Validate Hosted Pool Information (1ES PT)	Log
2786514	dotnet-aspnetcore	Run build.cmd	Log
2786499	dotnet-docker-tools	🛡 Guardian: Publish SDL Logs to Artifacts	Log
2786498	dotnet-docker-tools	🟦 Generate EOL Annotations - msdotnetcontainersstagingtest.azurecr.io	Log
2786466	dotnet-runtime	Send job to Helix (Unix)	Log
2786456	dotnet-dotnet	Build (centOSStreamContainer)	Log
2786328	dotnet-runtime	⏬ Pipeline Artifact Download	Log
1139378	dotnet/runtime	Send to Helix	Log
2786335	dotnet-dotnet-buildtools-prereqs-docker	🟪 Build Images	Log
2786313	dotnet-dotnet-buildtools-prereqs-docker	🛡 Generate SBoM Manifest	Log
Displaying 100 of 864 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
23	199	864

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez