The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
1347170	dotnet/maui	Controls Layout	Log	dotnet/maui#24776
1347139	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#29755
1347136	dotnet/maui	Run Integration Tests - Build windows	Log
1347132	dotnet/maui	Run Integration Tests - Build windows	Log
1347127	dotnet/maui	Run Integration Tests - Build windows	Log
1347131	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#34362
1347146	dotnet/maui	Controls CollectionView	Log	dotnet/maui#20488
1347124	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#34535
1347115	dotnet/maui	Run Integration Tests - Build windows	Log
2932892	dotnet-maui	Run Integration Tests - Build	Log
1347037	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#26641
1347038	dotnet/maui	Run Integration Tests - Build windows	Log
1347032	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#28474
1347033	dotnet/maui	Run Integration Tests - Build windows	Log
1347012	dotnet/maui	Run Integration Tests - Build windows	Log
1347011	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#28311
1347008	dotnet/aspnetcore	Run Quarantined Tests	Log
2932765	dotnet-roslyn-analyzers	Build and Test	Log
2932763	dotnet-roslyn-analyzers	Build and Test	Log
2932777	dotnet-roslyn-analyzers	Build and Test	Log
2932697	dotnet-docker-tools	🟪 Annotate EOL Images (msdotnetcontainersstaging.azurecr.io)	Log
2932677	dotnet-sdk	Run AoT Tests	Log	#59235
2932676	dotnet-sdk	Run AoT Tests	Log	#59217
1346796	dotnet/runtime	Send tests to Helix (Unix)	Log
2932672	dotnet-roslyn	Test	Log
1346726	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#125615
2932636	dotnet-runtime	Send job to Helix (Windows)	Log
2932587	dotnet-dotnet-buildtools-prereqs-docker	🟪 Build Images	Log
1346758	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log
2932612	dotnet-runtime	Send job to Helix (Unix)	Log
1346583	dotnet/runtime	Send to Helix	Log
2932583	dotnet-aspnetcore	Run build.cmd helix target	Log
2932577	dotnet-xharness	Generate SBOM manifest	Log
2932569	dotnet-aspnetcore	Run build.cmd helix target	Log
1346672	dotnet/maui	Run Integration Tests - Build windows	Log
1346675	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#29755
1346666	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#30089
1346667	dotnet/maui	Run Integration Tests - Build windows	Log
1346661	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#30538
1346662	dotnet/maui	Run Integration Tests - Build windows	Log
1346657	dotnet/maui	Run Integration Tests - Build windows	Log
1346656	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#32314
1346651	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#27847
1346652	dotnet/maui	Run Integration Tests - Build windows	Log
1346640	dotnet/maui	Run Integration Tests - Build windows	Log
1346658	dotnet/maui	Controls SafeAreaEdges,Shadow	Log
2932524	dotnet-performance	Run performance job script	Log
1346507	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#29755
1346504	dotnet/maui	Run Integration Tests - Build windows	Log
1346473	dotnet/maui	Run Integration Tests - Build windows	Log
1346472	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33179
1346467	dotnet/maui	Run Integration Tests - Build windows	Log
1346440	dotnet/maui	Run Integration Tests - Build windows	Log
1346306	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#30068
2932294	dotnet-dotnet-buildtools-prereqs-docker	🟪 Build Images	Log
1346298	dotnet/maui	Run Integration Tests - Build windows	Log
1346307	dotnet/maui	Run Integration Tests - Build windows	Log
1346297	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33451
2932325	dotnet-runtime	Send job to Helix (Unix)	Log
1346288	dotnet/maui	Run Integration Tests - Build windows	Log
1346280	dotnet/maui	Run Integration Tests - Build windows	Log
1346249	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#30047
1346250	dotnet/maui	Run Integration Tests - Build windows	Log
1346244	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#29814
1346245	dotnet/maui	Run Integration Tests - Build windows	Log
2932277	dotnet-aspnetcore	Run build.cmd helix target	Log
2932274	dotnet-aspnetcore	Run build.cmd helix target	Log
2932022	dotnet-dotnet-buildtools-prereqs-docker	🟪 Set Build Registry Options	Log
2932247	dotnet-dotnet	Build (centOSStreamContainer)	Log
2932226	dotnet-runtime	Send to Helix	Log
1346157	dotnet/runtime	AOT compile CoreCLR tests	Log
2932182	dotnet-runtime	Send job to Helix (Unix)	Log
2932114	dotnet-docker-tools	🟪 Annotate EOL Images (msdotnetcontainersstaging.azurecr.io)	Log
2932134	dotnet-sdk	Run AoT Tests	Log	#59217
2932135	dotnet-sdk	Run AoT Tests	Log	#59282
1346003	dotnet/maui	Controls CollectionView	Log
2932103	dotnet-roslyn	Test	Log
1346067	dotnet/runtime	Send tests to Helix (Unix)	Log
1346095	dotnet/aspnetcore	Run Quarantined Tests	Log
2932088	dotnet-roslyn	Test	Log
2932058	dotnet-roslyn	Test	Log
1346013	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log
2931956	dotnet-maui	Run Integration Tests - RunOniOS_MauiNativeAOT	Log
2932030	dotnet-roslyn	Test	Log
2931090	dotnet-maui	Run Integration Tests - Build	Log
2931917	dotnet-maui	Run Integration Tests - Build	Log
2931957	dotnet-maui	Controls (v18.5)	Log
1345908	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#26475
1345912	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#29755
1345909	dotnet/maui	Run Integration Tests - Build windows	Log
2931961	dotnet-dotnet	Build (centOSStreamContainer)	Log
1345884	dotnet/maui	Run Integration Tests - Build windows	Log
1345851	dotnet/maui	Run Integration Tests - Build windows	Log
2931982	dotnet-aspnetcore	Run build.cmd helix target	Log
2931976	dotnet-aspnetcore	Run build.cmd helix target	Log
1345817	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#125875
1345881	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#26213
1345937	dotnet/performance	Send job to Helix (Windows)	Log
1345763	dotnet/runtime	Send to Helix	Log	dotnet/runtime#125827
1345945	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#53464
Displaying 100 of 2105 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
47	678	2057

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez