NXP backend: Resolve limitations of uncertain tensor formats. #14576

MartinPavella · 2025-09-25T08:01:36Z

Summary

This PR resolves format related issues by inferring the format (NCHW/NHWC) for all nodes before partitioning. These formats are then used by the NeutronPartitioner to accurately determine which nodes are supported on Neutron.

Test plan

Unit tests provided, and correct function is tested by nearly every test in the nxp backend.

cc @kimishpatel

pytorch-bot · 2025-09-25T08:01:40Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14576

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 7 New Failures, 2 Unrelated Failures

As of commit 768c489 with merge base fdfeaa4 ():

NEW FAILURES - The following jobs have failed:

pull / test-samsung-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t 3e2a4a41eeda610bd724c480151091a3ee9970c2283cc0f71d4705b5be39950b /exec failed with exit code 1
Test CUDA Builds / export-gemma3-cuda-non-quantized / linux-job (gh)
RuntimeError: Command docker exec -t 1ed1ad6f2dd002ba11ba6abf97e7a0c2c6333148118b6b87144335bf9faa0dd3 /exec failed with exit code 2
Test CUDA Builds / export-gemma3-cuda-quantized-int4-tile-packed / linux-job (gh)
RuntimeError: Command docker exec -t 3c38c1f71505ab9dd58e647e6a39a801fc94d3f5a7736b6412d54523750b7826 /exec failed with exit code 2
Test CUDA Builds / export-voxtral-cuda-non-quantized / linux-job (gh)
RuntimeError: Command docker exec -t cbeaf73f4f1b606e93fa8b9f7061fa564d9605bf2672d582e11d0ed81abbcb5a /exec failed with exit code 2
Test CUDA Builds / export-voxtral-cuda-quantized-int4-tile-packed / linux-job (gh)
RuntimeError: Command docker exec -t 89a493246539287f9081330fb5b52a46c8e1fbe6733a7992b3b41e52c72289c2 /exec failed with exit code 2
Test CUDA Builds / export-voxtral-cuda-quantized-int4-weight-only / linux-job (gh)
RuntimeError: Command docker exec -t d1adb1d46cfe306f9067bc682828a71afd0f61c37c0cf3d13f313db514c18699 /exec failed with exit code 2
Test Metal Backend / export-voxtral-metal-artifact / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2

FLAKY - The following job failed but was likely due to flakiness present on trunk:

pull / unittest / linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / unittest-editable / linux / linux-job (gh) (trunk failure)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

MartinPavella · 2025-09-25T08:01:53Z

@pytorchbot label "module: nxp" "release notes: nxp"

kimishpatel · 2025-09-26T03:03:02Z

backends/nxp/TARGETS

    deps = [
        ":neutron_sdk",
        ":aten_passes",
+        ":_passes",


Can you add neutron_backend build to this https://github.com/pytorch/executorch/blob/main/.ci/scripts/unittest-buck2.sh. Thats the one that was failing last time. This way we get signal on the pr itself

facebook-github-bot · 2025-09-26T03:05:59Z

@kimishpatel has imported this pull request. If you are a Meta employee, you can view this in D83311730.

kimishpatel · 2025-09-26T03:06:36Z

i have imported it internally. There is an nxp test failing as well

digantdesai

Review automatically exported from Phabricator review in Meta.

MartinPavella · 2025-10-01T12:59:12Z

@kimishpatel I have fixed the failing nxp test, and added the nxp backend build to the unittest-buck.sh. The build is now failing with:

1: From load at backends/nxp/TARGETS:4
2: File not found: shim_et//arvr/tools/build_defs/oxx_python.bzl

These lines were in the file already before my changes. Do you know what is causing the issue/how to fix it?

kimishpatel · 2025-10-03T02:57:41Z

@kimishpatel I have fixed the failing nxp test, and added the nxp backend build to the unittest-buck.sh. The build is now failing with:

1: From load at backends/nxp/TARGETS:4 2: File not found: shim_et//arvr/tools/build_defs/oxx_python.bzl

These lines were in the file already before my changes. Do you know what is causing the issue/how to fix it?

I guess it is triggering your build in oss and we have never really had this test in oss so its trying to pull in internal files. I would say roll back that change for now just to be able to land.

MartinPavella · 2025-10-03T08:35:50Z

@kimishpatel I have fixed the failing nxp test, and added the nxp backend build to the unittest-buck.sh. The build is now failing with:
1: From load at backends/nxp/TARGETS:4 2: File not found: shim_et//arvr/tools/build_defs/oxx_python.bzl
These lines were in the file already before my changes. Do you know what is causing the issue/how to fix it?

I guess it is triggering your build in oss and we have never really had this test in oss so its trying to pull in internal files. I would say roll back that change for now just to be able to land.

@kimishpatel thanks for the response. I have removed the build as you said. Can you please now verify that your internal build passes?

kimishpatel · 2025-10-05T19:35:38Z

doing

MartinPavella · 2025-10-09T06:29:01Z

doing

@kimishpatel any updates? Is the build passing?

MartinPavella · 2025-10-13T05:22:18Z

doing

@kimishpatel any updates? Is the build passing?

@kimishpatel pinging in case you missed this.

StrycekSimon

Please update the numMacs related comment. Did the remove_geititem_pass.py just pass the linter with such long lines?

backends/nxp/_passes/remove_getitem_pass.py

backends/nxp/backend/ir/converter/node_converters/ops_converters/cat_converter.py

backends/nxp/tests/test_node_format_inference.py

backends/nxp/backend/node_format_inference.py

backends/nxp/tests/ir/converter/node_converter/test_softmax_converter.py

backends/nxp/backend/node_format.py

MartinPavella · 2025-10-14T08:23:08Z

Please update the numMacs related comment. Did the remove_geititem_pass.py just pass the linter with such long lines?

Yes, the linter passed. I have shortened the lines now anyway.

MartinPavella · 2025-10-22T11:36:15Z

doing

@kimishpatel any updates? Is the build passing?

@kimishpatel pinging in case you missed this.

@kimishpatel any updates? This issue is blocking multiple other PRs. I really don't want to break your internal builds again, so it would be great to confirm they are passing before merging this.

kimishpatel · 2025-10-22T14:43:04Z

Sorry for the late response on this. Lets rebase and then merge it

… node format. The pass `RemoveGetItemPass` replaces a `max_pool2d_with_indices` node with a `max_pool2d` node, that doesn't require a GetItem afterward. The new operator must, however, preserve the original node format. Therefore, a copy of the pass was created in `backends/nxp/_passes`, where it was modified. The new directory was created, because the pass doesn't follow the `NeutronEdgePass` interface.

Before, the format inference was done during conversion to NeutronIR (after partitioning), so the partitioner didn't yet know the formats. Now, the partitioner has the format data, which can be used to accurately select nodes for delegation.

…ode formats.

MartinPavella · 2025-10-23T10:04:38Z

@kimishpatel The failing checks appear unrelated, but I still cannot merge the PR.

The "cuda" checks fail on huggingface:

huggingface-cli login --token
error: argument --token: expected one argument

The samsung check fails on curl:

curl -sS -G --location --fail --retry 3 -H 'apikey: ' --data-urlencode version=0.7 --data-urlencode 'os=Ubuntu 22.04' https://soc-developer.semiconductor.samsung.com/api/v1/resource/ai-litecore/download
Failed to extract download URL

The pull / unittest-editable / linux / linux-job (pull_request) failed while building:

ready: 4/4 workers/pytorch/executorch/backends/test/suite/reporting.py:152: PytestCollectionWarning: cannot collect test class 'TestCaseSummary'

And pull / unittest/ linux / linux-job (pull_request) was canceled for some reason.

MartinPavella requested a review from kimishpatel September 25, 2025 08:01

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 25, 2025

pytorch-bot bot added module: nxp Issues related to NXP Neutron NPU delegation and code under backends/nxp/ release notes: nxp Changes to the NXP Neutron backend delegate labels Sep 25, 2025

kimishpatel reviewed Sep 26, 2025

View reviewed changes

digantdesai approved these changes Sep 26, 2025

View reviewed changes

MartinPavella force-pushed the upstream/main-nxp/EIEX-392-resolve-limitations-of-uncertain-tensor-formats branch 2 times, most recently from bb48aad to e1893f5 Compare October 1, 2025 07:41

MartinPavella marked this pull request as draft October 1, 2025 12:22

MartinPavella force-pushed the upstream/main-nxp/EIEX-392-resolve-limitations-of-uncertain-tensor-formats branch 2 times, most recently from 2672277 to 1ecfe36 Compare October 3, 2025 07:15

MartinPavella marked this pull request as ready for review October 3, 2025 08:34

MartinPavella requested a review from robert-kalmar as a code owner October 3, 2025 08:34

StrycekSimon suggested changes Oct 14, 2025

View reviewed changes

MartinPavella force-pushed the upstream/main-nxp/EIEX-392-resolve-limitations-of-uncertain-tensor-formats branch from 1ecfe36 to 27444cf Compare October 14, 2025 08:26

MartinPavella force-pushed the upstream/main-nxp/EIEX-392-resolve-limitations-of-uncertain-tensor-formats branch from 27444cf to fa43ac2 Compare October 14, 2025 08:27

MartinPavella requested a review from StrycekSimon October 14, 2025 09:12

robert-kalmar approved these changes Oct 14, 2025

View reviewed changes

StrycekSimon approved these changes Oct 14, 2025

View reviewed changes

MartinPavella mentioned this pull request Oct 14, 2025

NXP backend: Add support for aten.permute_copy.default. #15099

Draft

MartinPavella force-pushed the upstream/main-nxp/EIEX-392-resolve-limitations-of-uncertain-tensor-formats branch 2 times, most recently from 974c05d to a3ba1bb Compare October 22, 2025 11:34

MartinPavella added 7 commits October 23, 2025 10:55

NXP backend: Store inferred node format in the node.meta.

9e78e44

NXP backend: Improve cat delegation by using inferred node formats.

5c1d757

NXP backend: Improve constant_pad_nd delegation by using inferred n…

a3abfc2

…ode formats.

NXP backend: Do not infer format for unknown nodes.

aa8a8b9

NXP backend: Fix batch norm tests.

768c489

MartinPavella force-pushed the upstream/main-nxp/EIEX-392-resolve-limitations-of-uncertain-tensor-formats branch from a3ba1bb to 768c489 Compare October 23, 2025 08:56

NXP backend: Resolve limitations of uncertain tensor formats. #14576

Are you sure you want to change the base?

NXP backend: Resolve limitations of uncertain tensor formats. #14576

Uh oh!

Conversation

MartinPavella commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

pytorch-bot bot commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14576

❌ 7 New Failures, 2 Unrelated Failures

Uh oh!

MartinPavella commented Sep 25, 2025

Uh oh!

kimishpatel Sep 26, 2025

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Sep 26, 2025

Uh oh!

kimishpatel commented Sep 26, 2025

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

MartinPavella commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kimishpatel commented Oct 3, 2025

Uh oh!

MartinPavella commented Oct 3, 2025

Uh oh!

kimishpatel commented Oct 5, 2025

Uh oh!

MartinPavella commented Oct 9, 2025

Uh oh!

MartinPavella commented Oct 13, 2025

Uh oh!

StrycekSimon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MartinPavella commented Oct 14, 2025

Uh oh!

MartinPavella commented Oct 22, 2025

Uh oh!

kimishpatel commented Oct 22, 2025

Uh oh!

MartinPavella commented Oct 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

MartinPavella commented Sep 25, 2025 •

edited

Loading

pytorch-bot bot commented Sep 25, 2025 •

edited

Loading

MartinPavella commented Oct 1, 2025 •

edited

Loading