[arm][ez] Add xfail for norm tests

[SS-JIA]

Summary:

Context

#9938 made it so that linalg_vector_norm is now decomposed when exporting to Edge. However, this broke some tests in the arm delegate because export passes cannot handle the decomposed operator sequence. To account for this, add xfail for the failing tests since linalg_vector_norm is not supported in TOSA yet.

Changes

Add xfail for norm tests in test_torch_functions.py

Test Plan:

Test Plan

Check CI that failing test is recovered.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10043

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 89f3ff0 with merge base c9c5481 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[digantdesai]

Thank you!

[SS-JIA]

After investigating the root cause of the new test failures, my understanding of the situation is:

• linalg_vector_norm is not supported by TOSA and isn't lowered to the arm delegate. See the test output below, I collected it by running pytest backends/arm/test/models/test_torch_functions.py -s after reverting [ez][release blocker fix] Insert linalg_vector_norm into decomp table used for Edge export #9938
• After linalg_vector_norm is decomposed, it seems that the arm delegate export passes cannot handle the decomposed op sequence. The export pass that's having issues is the KeepDimsFalseToSqueezePass. Currently it seems that only pow is lowered after the op is decomposed.
• However, if linalg_vector_norm is added to the list of ops to not be decomposed, this will trigger a failure in _sanity_check_graph_for_non_decomp_ops() because the op is not decomposed and it is not lowered to arm delegate

It seems to me that it isn't correct to add linalg_vector_norm to the list of ops to not be decomposed, since arm delegate doesn't actually support it. The more correct thing to do, imo is add an xfail to the failing tests. In the future, linalg_vector_norm can be supported by supporting the op outright (at which point it can be added to the not decomposed list) or by supporting the decomposed op sequence.

Updating the PR to add xfail instead of not decomposing linalg_vector_norm. cc: @digantdesai

================================================================================================================================ test session starts ================================================================================================================================
platform linux -- Python 3.10.0, pytest-8.0.1, pluggy-1.4.0
rootdir: /home/ssjia/Github/executorch/backends/arm/test
configfile: pytest.ini
plugins: anyio-4.6.0, hypothesis-6.98.10, xdist-3.5.0, rerunfailures-15.0
collected 2 items                                                                                                                                                                                                                                                                   

backends/arm/test/models/test_torch_functions.py  ARM_TEST_SEED=273800142  INFO:executorch.backends.arm.test.tester.test_pipeline:Running pipeline with stages:
 ['export', 'check_not.quant_nodes', 'to_edge_transform_and_lower', 'check_not.exir', 'to_executorch', 'run_method_and_compare_outputs'].
INFO:executorch.backends.arm.tosa_partitioner:TOSAPartitioner::partition
INFO:executorch.backends.arm.tosa_partitioner:Partitioning for TOSABackend: TOSA-0.80+MI
INFO:executorch.backends.arm.tosa_partitioner:The following nodes were rejected for TOSA-0.80+MI:
INFO:executorch.backends.arm.tosa_partitioner:
╒═════════════════════════════════╤═════════════════════════════════╤═════════════════════════════╤═════════════════════════════════════╕
│ Node name                       │ Target                          │ Torch func                  │ Reason                              │
╞═════════════════════════════════╪═════════════════════════════════╪═════════════════════════════╪═════════════════════════════════════╡
│ aten_linalg_vector_norm_default │ aten.linalg_vector_norm.default │ ('norm_1', 'function.norm') │ Not included in BaseTOSASupportList │
│                                 │                                 │                             │ or a registered tosa_support_check  │
╘═════════════════════════════════╧═════════════════════════════════╧═════════════════════════════╧═════════════════════════════════════╛
INFO:executorch.backends.arm.tosa_partitioner:(Placeholders and outputs are not included in this list)
INFO:executorch.backends.arm.test.tester.arm_tester:Comparing Stage 'ToExecutorch' with Stage 'InitialModel'
INFO:executorch.backends.arm.test.tester.arm_tester:Run #0, input shapes: [5, 5]
ERROR:executorch.backends.arm.test.tester.test_pipeline:
Failure in stage <run_method_and_compare_outputs>: 
   Ran model with TosaReferenceModelDispatch but never ran TOSABackend delegate.
. ARM_TEST_SEED=4240174922  INFO:executorch.backends.arm.test.tester.test_pipeline:Running pipeline with stages:
 ['quantize', 'export', 'to_edge_transform_and_lower', 'check_not.exir', 'to_executorch', 'run_method_and_compare_outputs'].
INFO:executorch.backends.arm.tosa_partitioner:TOSAPartitioner::partition
INFO:executorch.backends.arm.tosa_partitioner:Partitioning for TOSABackend: TOSA-0.80+BI
INFO:executorch.backends.arm.tosa_partitioner:The following nodes were rejected for TOSA-0.80+BI:
INFO:executorch.backends.arm.tosa_partitioner:
╒═════════════════════════════════╤═════════════════════════════════╤═════════════════════════════╤═════════════════════════════════════╕
│ Node name                       │ Target                          │ Torch func                  │ Reason                              │
╞═════════════════════════════════╪═════════════════════════════════╪═════════════════════════════╪═════════════════════════════════════╡
│ aten_linalg_vector_norm_default │ aten.linalg_vector_norm.default │ ('norm_1', 'function.norm') │ Not included in BaseTOSASupportList │
│                                 │                                 │                             │ or a registered tosa_support_check  │
╘═════════════════════════════════╧═════════════════════════════════╧═════════════════════════════╧═════════════════════════════════════╛
INFO:executorch.backends.arm.tosa_partitioner:(Placeholders and outputs are not included in this list)
INFO:executorch.backends.arm.test.tester.arm_tester:Comparing Stage 'ToExecutorch' with Stage 'Quantize'
INFO:executorch.backends.arm.test.tester.arm_tester:Run #0, input shapes: [5, 5]
ERROR:executorch.backends.arm.test.tester.test_pipeline:
Failure in stage <run_method_and_compare_outputs>: 
   Ran model with TosaReferenceModelDispatch but never ran TOSABackend delegate.
.

[zingo]

Thanks for the fix, good work!

[SS-JIA]

@pytorchbot cherry-pick --onto release/0.6 -c critical

[pytorchbot]

Cherry picking #10043

The cherry pick PR is at #10078 and it is recommended to link a critical cherry pick PR with an issue. The following tracker issues are updated:

• [v0.6.0] Release Tracker #9253 (comment)

Details for Dev Infra team

Raised by workflow job