Revert D76283309: Multisect successfully blamed "D76000131: [pytorch/executorch][diff_train] Support exp op in XNNPACK backend (#11373)" for 100+ test failures #11530

leafs1 · 2025-06-10T17:40:32Z

Summary:
This diff reverts D76283309
D76000131: [pytorch/executorch][diff_train] Support exp op in XNNPACK backend (#11373) by leafs1 causes the following test failures:

Depends on D76283309

Differential Revision: D76331094

…executorch][diff_train] Support exp op in XNNPACK backend (pytorch#11373)" for 100+ test failures Summary: This diff reverts D76283309 D76000131: [pytorch/executorch][diff_train] Support exp op in XNNPACK backend (pytorch#11373) by leafs1 causes the following test failures: Tests affected: - [fbsource//xplat/mantle/runner/v2/image_concept_detection_node_runner:scenex_model_testApple - SceneXModelTest/TestDataProcess](https://www.internalfb.com/intern/test/844425100853050/) - [fbsource//xplat/mantle/runner/v2/image_concept_detection_node_runner:image_concept_detection_node_runner_testAndroid-32bit - runAllTests (com.facebook.xplat.XplatTestRunner)](https://www.internalfb.com/intern/test/844425103485141/) - [fbsource//fbobjc/Libraries/FBMLModelRunnerKit/Models/SceneX:SceneXTests - SceneXTests.SceneXModelTests/testSceneXv5WithExecutorch](https://www.internalfb.com/intern/test/844425103573097/) - [fbsource//xplat/mantle/runner/v2/image_concept_detection_node_runner:image_concept_detection_node_runner_testApple - ImageConceptDetectionNodeRunnerTest/TestFailingGate](https://www.internalfb.com/intern/test/844425108858774/) - [fbsource//xplat/mantle/runner/v2/entity_detection_node_runner:entity_detection_provider_testAndroid - main](https://www.internalfb.com/intern/test/844425109166703/) - [fbsource//xplat/mantle/runner/v2/entity_detection_node_runner:entity_detection_provider_testApple - EntityDetectionProviderTest/TestInference](https://www.internalfb.com/intern/test/844425110282954/) - [fbsource//xplat/mantle/runner/v2/entity_detection_node_runner:entity_detection_node_runner_testAppleMac - EntityDetectionNodeRunnerTest/TestTriggerModel](https://www.internalfb.com/intern/test/844425111066260/) - [fbsource//xplat/executorch/extension/fb/dynamic_shim/test:whatsapp_bwe_dynamic_shim_testAppleMac - DynamicShimTest/WhatsAppBwe](https://www.internalfb.com/intern/test/844425111857885/) - [fbsource//xplat/assistant/integration_test/tests/supernova/messaging:message_privacy_tests - MessagingTests/MessagePrivacyTests.ContactResolvedWithExplicitProvider/WhatsApp](https://www.internalfb.com/intern/test/844425113464690/) - [fbsource//xplat/assistant/integration_test/tests/supernova/messaging:message_privacy_tests - MessagingTests/MessagePrivacyTests.ContactResolved/Native](https://www.internalfb.com/intern/test/844425113464691/) ...and 90 more Here's the Multisect link: https://www.internalfb.com/multisect/29241293 Here are the tasks that are relevant to this breakage: T227056473: Your diff, D76000131, broke 500+ tests T168807700: 100+ tests, 50+ build rules, 10+ CI signals, some sandcastle jobs, some skycastle workflows failing for ai_infra_mobile_platform T191382063: 10+ tests, some CI signals failing for messenger_relevance_delivery_infra T227057770: Test fbsource//fbobjc/Libraries/FBMLModelRunnerKit/ModelRuntimes/Executorch/ExecutorchRuntimeSceneXTests:ExecutorchRuntimeSceneXTestsTests - ExecutorchRuntimeSceneXTestsTests.ExecutorchRuntimeSceneXTe T227058008: Test fbsource//fbobjc/Libraries/FBMLModelRunnerKit/ModelRuntimes/Executorch/ExecutorchRuntimeSceneXV2Tests:ExecutorchRuntimeSceneXV2TestsTests - ExecutorchRuntimeSceneXV2TestsTests.ExecutorchRuntimeSc T227058748: Test fbsource//fbobjc/Libraries/FBMLModelRunnerKit/Models/SceneX:SceneXTests - SceneXTests.SceneXModelTests/testSceneXv5WithExecutorch failing for inspiration_sharing_foundation_ios T227058778: Test fbsource//fbobjc/Libraries/ReadyMadeSharing/AdvancedPro/FBAdvancedProSceneX:FBAdvancedProSceneXTests - FBAdvancedProSceneXTests.FBAdvancedProSceneXTests/testSceneXv5WithExecutorchAdvancedPro fail T220012969: 10+ tests unhealthy for fb_video_expression_ios T227068901: Test fbsource//xplat/assistant/integration_test/tests/supernova/messaging:message_privacy_tests - MessagingTests/MessagePrivacyTests.ContactNotResolved/MessengerE2EE failing for ar_ai_runtime_core T227069031: Test fbsource//xplat/assistant/integration_test/tests/supernova/messaging:message_privacy_tests - MessagingTests/MessagePrivacyTests.ContactNotResolved/MessengerEPD failing for ar_ai_runtime_core T227068897: Test fbsource//xplat/assistant/integration_test/tests/supernova/messaging:message_privacy_tests - MessagingTests/MessagePrivacyTests.ContactNotResolvedWithExplicitProvider/WhatsApp failing for ar_ai_r T227068838: Test fbsource//xplat/assistant/integration_test/tests/supernova/messaging:message_privacy_tests - MessagingTests/MessagePrivacyTests.ContactResolvedWithExplicitProvider/WhatsApp failing for ar_ai_runt T227069028: Test fbsource//xplat/assistant/integration_test/tests/supernova/messaging:message_privacy_tests - MessagingTests/MessagePrivacyTests.ContactNotResolved/Native failing for ar_ai_runtime_core T227068840: Test fbsource//xplat/assistant/integration_test/tests/supernova/messaging:message_privacy_tests - MessagingTests/MessagePrivacyTests.ContactResolved/Native failing for ar_ai_runtime_core T227068812: Test fbsource//xplat/assistant/integration_test/tests/supernova/messaging:message_privacy_tests - MessagingTests/MessagePrivacyTests.ContactNotResolved/WhatsApp failing for ar_ai_runtime_core T227068896: Test fbsource//xplat/assistant/integration_test/tests/supernova/messaging:message_privacy_tests - MessagingTests/MessagePrivacyTests.ContactResolvedWithExplicitProvider/MessengerEPD failing for ar_ai_ T227068966: Test fbsource//xplat/assistant/integration_test/tests/supernova/messaging:message_privacy_tests - MessagingTests/MessagePrivacyTests.ContactResolvedWithExplicitProvider/Native failing for ar_ai_runtim T227068965: Test fbsource//xplat/assistant/integration_test/tests/supernova/messaging:message_privacy_tests - MessagingTests/MessagePrivacyTests.ContactNotResolvedWithExplicitProvider/Native failing for ar_ai_run T227068902: Test fbsource//xplat/assistant/integration_test/tests/supernova/messaging:it_it_messaging_golden_tests - ItItMessaging/MessagingTestsHeader.BasicTest/messaging_native___it_IT failing for ar_ai_runtime The backout may land if someone accepts it. Revert reason: 100 total blames is over revert threshold of 5 for app messenger_for_ios per SV_RELENG_MULTISECT_AUTO_REVERT_CONFIG. If this diff has been generated in error, you can Commandeer and Abandon it. Depends on D76283309 Differential Revision: D76331094

pytorch-bot · 2025-06-10T17:40:36Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/11530

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 9 New Failures, 61 Unrelated Failures

As of commit 6a43cf0 with merge base 3f11883 ():

NEW FAILURES - The following jobs have failed:

Build documentation / build (buck2) / Build doc (gh)
The process '/usr/bin/git' failed with exit code 128
Build Presets / apple (ios-simulator) / build (gh)
The process '/opt/homebrew/bin/git' failed with exit code 1
Build Presets / apple (ios) / build (gh)
The process '/opt/homebrew/bin/git' failed with exit code 1
Build Presets / apple (llm) / build (gh)
The process '/opt/homebrew/bin/git' failed with exit code 1
Build Presets / apple (macos) / build (gh)
The process '/opt/homebrew/bin/git' failed with exit code 1
Build Presets / apple (pybind) / build (gh)
The process '/opt/homebrew/bin/git' failed with exit code 1
Build Presets / linux (linux, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
The process '/usr/bin/git' failed with exit code 128
Build Presets / linux (pybind, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
The process '/usr/bin/git' failed with exit code 128
pull / android / build-llm-demo / linux-job (gh)
The process '/usr/bin/git' failed with exit code 1

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

pull / test-binary-size-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-binary-size-linux-gcc / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-custom-ops-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-eval_llama-mmlu-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-eval_llama-wikitext-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-llama_runner_eager-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-llama-runner-linux (bf16, custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-llama-runner-linux-android / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-llava-runner-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-mediatek-models-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-models-linux (add_mul, portable, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (add_mul, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-models-linux (add, portable, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (add, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (emformer_join, portable, linux.4xlarge.memory) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-models-linux (emformer_join, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-models-linux (emformer_transcribe, portable, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (ic3, portable, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (ic3, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (ic4, portable, linux.4xlarge.memory) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-models-linux (linear, portable, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (linear, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (mobilebert, portable, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (mobilebert, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (mv2, portable, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (mv2, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (resnet18, portable, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (resnet18, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (resnet50, portable, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (resnet50, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux (w2l, portable, linux.4xlarge.memory) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux-basic (mv3, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux-basic (mv3, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux-basic (vit, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux-basic (vit, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-moshi-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-openvino-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-phi-3-mini-runner-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-qnn-models-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-quantized-aot-lib-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-selective-build-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / test-setup-linux-gcc / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
pull / test-static-llama-qnn-linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / unittest / linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / unittest-buck / linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
pull / unittest-editable / linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-06-10T17:40:43Z

This pull request was exported from Phabricator. Differential Revision: D76331094

github-actions · 2025-06-10T17:41:22Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

digantdesai · 2025-06-25T20:15:41Z

Is this PR still relevant?

leafs1 requested review from digantdesai and mcr229 as code owners June 10, 2025 17:40

pytorch-bot bot added the ci-no-td label Jun 10, 2025

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 10, 2025

facebook-github-bot added the fb-exported label Jun 10, 2025

leafs1 closed this Jun 25, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Revert D76283309: Multisect successfully blamed "D76000131: [pytorch/executorch][diff_train] Support exp op in XNNPACK backend (#11373)" for 100+ test failures #11530

Revert D76283309: Multisect successfully blamed "D76000131: [pytorch/executorch][diff_train] Support exp op in XNNPACK backend (#11373)" for 100+ test failures #11530

Uh oh!

leafs1 commented Jun 10, 2025 •

edited by mcr229

Loading

Uh oh!

pytorch-bot bot commented Jun 10, 2025 •

edited

Loading

Uh oh!

facebook-github-bot commented Jun 10, 2025

Uh oh!

github-actions bot commented Jun 10, 2025

Uh oh!

digantdesai commented Jun 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Revert D76283309: Multisect successfully blamed "D76000131: [pytorch/executorch][diff_train] Support exp op in XNNPACK backend (#11373)" for 100+ test failures #11530

Revert D76283309: Multisect successfully blamed "D76000131: [pytorch/executorch][diff_train] Support exp op in XNNPACK backend (#11373)" for 100+ test failures #11530

Uh oh!

Conversation

leafs1 commented Jun 10, 2025 • edited by mcr229 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/11530

❌ 9 New Failures, 61 Unrelated Failures

Uh oh!

facebook-github-bot commented Jun 10, 2025

Uh oh!

github-actions bot commented Jun 10, 2025

This PR needs a release notes: label

Uh oh!

digantdesai commented Jun 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

leafs1 commented Jun 10, 2025 •

edited by mcr229

Loading

pytorch-bot bot commented Jun 10, 2025 •

edited

Loading

This PR needs a `release notes:` label