Skip to content

NXP backend: Add MobileNetV2 example model and test #12892

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
robert-kalmar merged 3 commits into from
Aug 26, 2025
Merged

NXP backend: Add MobileNetV2 example model and test #12892

robert-kalmar merged 3 commits into from
Aug 26, 2025

Conversation

@StrycekSimon
Copy link
Collaborator

@StrycekSimon StrycekSimon commented Jul 28, 2025 •
edited by robert-kalmar
Loading

Summary

Add MobileNetV2 model as example and for integration testing

Test plan

Support for testing full conversion on this model is included in run_aot_example.sh.

cc @skywall @robert-kalmar

@pytorch-bot
Copy link

pytorch-bot bot commented Jul 28, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12892

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Pending

As of commit 39c59c8 with merge base 49bc664 (image):

NEW FAILURE - The following job has failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 28, 2025
@StrycekSimon
Copy link
Collaborator Author

StrycekSimon commented Jul 28, 2025

@pytorchbot label "release notes: nxp"

@pytorch-bot pytorch-bot bot added the release notes: nxp Changes to the NXP Neutron backend delegate label Jul 28, 2025
@StrycekSimon StrycekSimon force-pushed the upstream/main-nxp/EIEX-396-upstream-toy-models-integration branch from c2dae16 to afb0257 Compare July 28, 2025 14:49
@robert-kalmar robert-kalmar self-requested a review August 4, 2025 08:10
robert-kalmar
robert-kalmar approved these changes Aug 4, 2025
View reviewed changes
Copy link
Collaborator

@robert-kalmar robert-kalmar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. The failing tests seems unrelated to this PR.

digantdesai
digantdesai approved these changes Aug 5, 2025
View reviewed changes
Copy link
Contributor

@digantdesai digantdesai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is just AOT but if you ran the PTE internally on your end what's the error looks like when you compare exported, quantized GraphModule on PyTorch vs. Neutron Simulator elementwise? i.e. ATOL, RTOL for tensor.all_close?

@robert-kalmar robert-kalmar force-pushed the upstream/main-nxp/EIEX-396-upstream-toy-models-integration branch from afb0257 to 670002b Compare August 26, 2025 09:07
@robert-kalmar
Copy link
Collaborator

robert-kalmar commented Aug 26, 2025

I know this is just AOT but if you ran the PTE internally on your end what's the error looks like when you compare exported, quantized GraphModule on PyTorch vs. Neutron Simulator elementwise? i.e. ATOL, RTOL for tensor.all_close?

We calculate and check the MSE, and the threshold is set to 8.6e-2 (note the output is FP32)

Here is the result from CPU and NPU run:

CPU:

example_cassette_player482_2_2.bin 848 5.412652 
example_chain_saw491_3_3.bin 491 5.152845 
example_French_horn566_5_5.bin 566 5.455954 
example_bench703_0_0.bin 0 5.152845 
example_gas_pump571_7_7.bin 480 5.629158 
example_garbage_truck569_6_6.bin 569 8.746846 
example_church497_4_4.bin 497 4.460026 
example_golf_ball574_8_8.bin 574 6.798292 
example_English_springer217_1_1.bin 217 5.759062 
example_parachute701_9_9.bin 701 6.148773

Neutron NPU

example_cassette_player482_2_2.bin 848 5.066243 
example_chain_saw491_3_3.bin 491 5.326050 
example_French_horn566_5_5.bin 566 5.109544 
example_bench703_0_0.bin 0 4.979640 
example_gas_pump571_7_7.bin 480 5.715761 
example_garbage_truck569_6_6.bin 569 8.746846 
example_church497_4_4.bin 497 4.763134 
example_golf_ball574_8_8.bin 574 6.625087 
example_English_springer217_1_1.bin 217 5.715761 
example_parachute701_9_9.bin 701 6.841593

@robert-kalmar robert-kalmar force-pushed the upstream/main-nxp/EIEX-396-upstream-toy-models-integration branch from 670002b to 39c59c8 Compare August 26, 2025 09:48
@robert-kalmar robert-kalmar merged commit 71206cf into pytorch:main Aug 26, 2025
109 of 110 checks passed
@robert-kalmar robert-kalmar deleted the upstream/main-nxp/EIEX-396-upstream-toy-models-integration branch August 26, 2025 11:26
agrima1304 pushed a commit to agrima1304/executorch that referenced this pull request Aug 26, 2025
@StrycekSimon @skywall @agrima1304
NXP backend: Add MobileNetV2 example model and test (pytorch#12892)
36cdaec 
### Summary
Add MobileNetV2 model as example and for integration testing

### Test plan
Support for testing full conversion on this model is included in
`run_aot_example.sh`.

---------

Co-authored-by: Lukas Sztefek <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@digantdesai digantdesai digantdesai approved these changes

@robert-kalmar robert-kalmar robert-kalmar approved these changes

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. release notes: nxp Changes to the NXP Neutron backend delegate

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@StrycekSimon @robert-kalmar @digantdesai @facebook-github-bot @skywall