The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
2895271	dotnet-runtime	Send ZipArchiveFuzzer to OneFuzz	Log
1279452	dotnet/runtime	Send to Helix	Log
1279587	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#123500
1279699	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33890
1279464	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#123142
1279573	dotnet/runtime	Send to Helix	Log
1279467	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#123888
1279608	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#120326
2895351	dotnet-efcore	Send job to helix	Log
1279106	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33887
2895301	dotnet-dotnet	Build (centOSStreamContainer)	Log
1279449	dotnet/runtime	Send to Helix	Log
1278825	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#123478
1279363	dotnet/runtime	Send tests to Helix (Unix)	Log
2895288	dotnet-runtime	Send job to Helix (Windows)	Log
1279439	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#65320
1277913	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#65306
2895258	dotnet-performance	Send job to Helix (Unix)	Log
1279340	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#64644
1279324	dotnet/maui	Controls Layout	Log	dotnet/maui#33234
1279330	dotnet/runtime	Build product	Log	dotnet/runtime#123986
1279328	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#52596
1279283	dotnet/aspnetcore	Run build.cmd helix target	Log
2895174	dotnet-runtime	Send job to Helix (Unix)	Log
2895065	dotnet-dotnet	Build	Log
2895050	dotnet-runtime	Send job to Helix (Unix)	Log
2895013	dotnet-dotnet	Build (centOSStreamContainer)	Log
1278712	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#123926
1278875	dotnet/aspnetcore	Run Quarantined Tests	Log
1278876	maestro-auth-test/helix-queue-insights-test	Run Tests (Windows)	Log
1278746	dotnet/maui	Controls CollectionView	Log
2894911	dotnet-extensions	Set Release Configs Vars	Log
2894932	dotnet-performance	Send job to Helix (Unix)	Log
1278595	dotnet/runtime	Send tests to Helix (Unix)	Log
1278275	dotnet/maui	Controls Layout	Log
1278243	dotnet/maui	Controls Layout	Log	dotnet/maui#33686
1278756	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33883
2894901	dotnet-runtime	Send job to Helix (Unix)	Log
2894645	dotnet-maui	Controls (v18.4) WebView	Log
2894843	dotnet-dotnet	Build	Log
1278621	dotnet/aspnetcore	Run Quarantined Tests	Log
1278451	dotnet/runtime	Build native test components	Log	dotnet/runtime#123961
2894644	dotnet-maui	Run Integration Tests - Build	Log
2894783	dotnet-dotnet	Build (centOSStreamContainer)	Log
2894531	dotnet-maui	Controls CollectionView	Log
1278216	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#123412
1271213	dotnet/maui	Run Integration Tests - RunOniOS_BlazorDebug macOS	Log
1278212	dotnet/maui	Controls (v18.4) Page,Performance,Picker,ProgressBar	Log	dotnet/maui#33850
2894792	dotnet-runtime	Install workload using artifacts	Log
1278434	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#52792
1278248	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#122556
1278049	dotnet/maui	Controls CollectionView	Log
1278277	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log
2894723	dotnet-dotnet	Build (centOSStreamContainer)	Log
2894758	dotnet-sdk	Run Tests in Helix	Log	#57507
2894759	dotnet-sdk	Run Tests in Helix	Log	#57508
1278345	dotnet/machinelearning	Run Helix Tests	Log	dotnet/machinelearning#7575
1278410	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33834
2894676	dotnet-installer	Build	Log	#57500
1278347	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123657
2894708	dotnet-dotnet	Build	Log	#57506
2894648	dotnet-runtime	Send job to Helix (Unix)	Log
1278187	dotnet/runtime	Build product	Log	dotnet/runtime#123515
1278274	dotnet/maui	Run Integration Tests - Build windows	Log
2894635	dotnet-aspire	Initialize containers	Log
2894066	dotnet-aspnetcore	Run build.cmd helix target	Log	#57456
2894626	dotnet-performance	Send job to Helix (Unix)	Log
1278061	dotnet/maui	Controls CollectionView	Log
1278158	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33863
2894384	dotnet-maui	Controls Layout	Log
1278053	dotnet/maui	Controls Layout	Log
1277966	dotnet/runtime	Publish	Log	dotnet/runtime#123742
2894510	dotnet-dotnet	Build (centOSStreamContainer)	Log
1278025	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123408
2894530	dotnet-maui	Run Integration Tests - RunOniOS_MauiNativeAOT	Log
1277754	dotnet/maui	Controls Layout	Log	dotnet/maui#33856
1277990	dotnet/maui	Run Integration Tests - RunOniOS_MauiNativeAOT macOS	Log
2894479	dotnet-maui	Controls Layout	Log
2894339	dotnet-maui	Controls (v18.4) ListView	Log
1277706	dotnet/maui	Run Integration Tests - Build windows	Log
1277991	dotnet/maui	Controls CollectionView	Log
1277928	dotnet/maui	Run Integration Tests - RunOniOS_MauiNativeAOT macOS	Log	dotnet/maui#33838
1277963	dotnet/maui	Run Integration Tests - Build windows	Log
2894313	dotnet-maui	Controls (v18.4) TabbedPage,TableView,TimePicker,TitleView,ToolbarItem	Log
1277711	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log	dotnet/maui#33856
2894275	dotnet-runtime	Send ZipArchiveFuzzer to OneFuzz	Log
2894439	dotnet-aspnetcore	Run build.cmd helix target	Log	#57410
1277898	dotnet/maui	Controls CollectionView	Log
1277800	dotnet/maui	Controls Layout	Log
1277799	dotnet/maui	Run Integration Tests - Build windows	Log
1277748	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log	dotnet/maui#33524
1277705	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log
1277771	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33858
1277659	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#121906
1274496	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33524
1277724	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33846
2894312	dotnet-maui	Run Integration Tests - Build	Log
1277709	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33856
1277626	dotnet/runtime	Send tests to Helix (Unix)	Log
1277703	dotnet/maui	Run Integration Tests - Build windows	Log
Displaying 100 of 2510 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
69	654	2510

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez