The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Repository	Step Name	Console log	Pull Request
1545728	dotnet/aspnetcore	Run Quarantined Tests	Log
1545673	dotnet/aspnetcore	Run Quarantined Tests	Log
1545566	dotnet/maui	Controls Shell	Log	dotnet/maui#35706
1545550	dotnet/maui	Controls (vlatest) Entry	Log	dotnet/maui#37049
1545289	dotnet/maui	Controls Shell	Log
1545422	dotnet/maui	Controls Shell	Log
1545385	dotnet/aspnetcore	Run Quarantined Tests	Log
1545183	dotnet/runtime	Send to Helix	Log
1545287	dotnet/aspnetcore	Run Quarantined Tests	Log
1545281	dotnet/roslyn	Run Unit Tests	Log
1545167	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#132028
1545135	dotnet/runtime	Send tests to Helix (Unix)	Log
1545184	dotnet/runtime	Send job to Helix (Unix)	Log
1544766	dotnet/maui	Controls CollectionView	Log
1545150	dotnet/aspnetcore	Run Quarantined Tests	Log
1544925	dotnet/runtime	Send to Helix	Log
1545088	dotnet/runtime	Send job to Helix (Unix)	Log
1544960	dotnet/runtime	Zip Build Assets	Log
1544923	dotnet/runtime	Send tests to Helix (Unix)	Log
1545012	dotnet/arcade	Run Helix Tests	Log	dotnet/arcade#17204
1544879	dotnet/roslyn	Run Unit Tests	Log
1544637	dotnet/runtime	Publish	Log	dotnet/runtime#131972
1544748	dotnet/runtime	Build product	Log	dotnet/runtime#132014
1544770	dotnet/runtime	Build product	Log	dotnet/runtime#132018
1544627	dotnet/runtime	Build product	Log	dotnet/runtime#131993
1544493	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#132002
1544653	dotnet/roslyn	Run Unit Tests	Log
1544596	dotnet/runtime	Build product	Log	dotnet/runtime#132008
1544592	dotnet/maui	Controls Shell	Log
1544490	dotnet/maui	Controls Layout	Log	dotnet/maui#36411
1544464	dotnet/roslyn	Run Unit Tests	Log
1544436	dotnet/maui	Controls Layout	Log	dotnet/maui#37147
1544469	dotnet/maui	Controls CollectionView	Log
1544443	dotnet/maui	Controls CollectionView	Log	dotnet/maui#37184
1544433	dotnet/maui	Controls CollectionView	Log	dotnet/maui#37182
1544272	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#131972
1544324	dotnet/maui	Controls Layout	Log
1544136	dotnet/runtime	Send tests to Helix (Unix)	Log
1544252	dotnet/maui	Controls ScrollView	Log	dotnet/maui#37110
1542775	dotnet/arcade	Run Helix Tests	Log	dotnet/arcade#17204
1544083	dotnet/runtime	Publish Build Assets	Log
1544138	dotnet/runtime	Build product	Log
1544080	dotnet/aspnetcore	Run Quarantined Tests	Log
1544034	dotnet/maui	Controls CollectionView	Log	dotnet/maui#37033
1544024	dotnet/roslyn	Run Unit Tests	Log
1543685	dotnet/runtime	Zip	Log	dotnet/runtime#131981
1543542	dotnet/runtime	Build native test components	Log	dotnet/runtime#131967
1543592	dotnet/runtime	Send to Helix	Log	dotnet/runtime#131740
1543407	dotnet/roslyn	Run Unit Tests	Log
1543309	dotnet/runtime	Send to Helix	Log
1543470	dotnet/sdk	Run AoT Tests	Log	dotnet/sdk#55643
1543469	dotnet/sdk	Run AoT Tests	Log	dotnet/sdk#55642
1543405	dotnet/maui	Controls Shell	Log	dotnet/maui#36511
1543206	dotnet/runtime	Send to Helix	Log
1543399	dotnet/roslyn	Run Unit Tests	Log
1543338	dotnet/maui	Controls CollectionView	Log
1543359	dotnet/roslyn	Run Unit Tests	Log
1543345	dotnet/roslyn	Run Unit Tests	Log
1543299	dotnet/aspnetcore	Run Quarantined Tests	Log
1543203	dotnet/runtime	Send to Helix	Log
1543009	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#131352
1543166	dotnet/maui	Controls Shell	Log
1542691	dotnet/runtime	Send to Helix	Log
1539701	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#131411
1542895	dotnet/aspnetcore	Run Quarantined Tests	Log
1542689	dotnet/runtime	Send tests to Helix (Unix)	Log
1542711	dotnet/maui	Controls CollectionView	Log
1542723	dotnet/maui	Controls Shell	Log	dotnet/maui#37148
1542715	dotnet/aspnetcore	Run Quarantined Tests	Log
1542726	dotnet/runtime	Build product	Log	dotnet/runtime#131202
1542511	dotnet/roslyn	Run Unit Tests	Log
1542485	dotnet/maui	Controls Layout	Log	dotnet/maui#37143
1542387	dotnet/roslyn	Run Unit Tests	Log
1542385	dotnet/roslyn	Run Unit Tests	Log
1542268	dotnet/runtime	Send to Helix	Log	dotnet/runtime#131927
1542374	dotnet/sdk	Run TestBuild Tests	Log	dotnet/sdk#55632
1542306	dotnet/aspnetcore	Run Quarantined Tests	Log
1542212	dotnet/maui	Controls (CV1)	Log	dotnet/maui#37057
1542153	dotnet/maui	Controls Shell	Log	dotnet/maui#36443
1542078	dotnet/runtime	Send to Helix	Log
1542090	dotnet/roslyn	Run Unit Tests	Log
1541745	dotnet/runtime	Send to Helix	Log
1541801	dotnet/runtime	Publish	Log	dotnet/runtime#131908
1541635	dotnet/runtime	Send to Helix	Log
1541749	dotnet/maui	Controls Shell	Log
1541764	dotnet/roslyn	Run Unit Tests	Log
1541556	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#131671
1541657	dotnet/roslyn	Run Unit Tests	Log
1541180	dotnet/runtime	Send to Helix	Log
1541064	dotnet/runtime	Send to Helix	Log	dotnet/runtime#131209
1541578	dotnet/aspnetcore	Run Quarantined Tests	Log
1541178	dotnet/runtime	Send tests to Helix (Unix)	Log
1541498	dotnet/runtime	Send job to Helix (Unix)	Log
1541420	dotnet/maui	Controls Shell	Log	dotnet/maui#37119
1541446	dotnet/sdk	Run AoT Tests	Log
1541362	dotnet/aspnetcore	Run Quarantined Tests	Log
1541375	dotnet/roslyn	Run Unit Tests	Log
1541138	dotnet/aspnetcore	Run Quarantined Tests	Log
1541134	dotnet/maui	Controls Shell	Log
1541080	dotnet/maui	Controls Shell	Log
Displaying 100 of 2220 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
7	234	2220

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez