Skip to content

Commit 1e4c316

Browse files
Guang Yangfacebook-github-bot
Guang Yang
authored and
facebook-github-bot
committed
Add more data points from benchmarking infra (#5432)
Summary: As titled Pull Request resolved: #5432 Reviewed By: kirklandsign Differential Revision: D62885076 Pulled By: guangy10 fbshipit-source-id: 31d3fbc270904a59a8983ce963c6ef831bc9bf89
1 parent 444480b commit 1e4c316

File tree

1 file changed

+12
-8
lines changed

1 file changed

+12
-8
lines changed

‎extension/benchmark/README.md‎

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -23,20 +23,24 @@ Below is a table summarizing some example data points obtained via the infra. Th
2323

2424
| Model | Backend | Model Load Time (ms) | Avg Inference Latency (ms) | Device |
2525
|-----------------------|--------------|----------------------|----------------------------|------------------------|
26-
| DeepLabV3 (dl3) | XNN Q8 | [90.616](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218985758) | [666.219](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218985758) | Samsung S22 |
27-
| DeepLabV3 (dl3) | QNN FP16 | [182.207](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218985141) | [9.759](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218985141) | Samsung S22 |
28-
| MobileNetV3 (mv3) | XNN Q8 | [34.024](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218988461) | [252.628](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218988461) | Samsung S22 |
26+
| MobileNetV3 (mv3) | XNNPACK Q8 | [34.024](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218988461) | [252.628](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218988461) | Samsung S22 |
2927
| MobileNetV3 (mv3) | QNN FP16 | [168.414](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218987785) | [1.182](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218987785) | Samsung S22 |
3028
| MobileNetV3 (mv3) | COREML FP16 | [61.583](https://github.com/pytorch/executorch/actions/runs/10836589365/job/30078681158) | [682.197](https://github.com/pytorch/executorch/actions/runs/10836589365/job/30078681158) | Apple iPhone 15 Pro |
31-
| MobileNetV2 (mv2) | XNN Q8 | [14.397](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218987379) | [10.796](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218987379) | Samsung S22 |
29+
| MobileNetV2 (mv2) | XNNPACK Q8 | [14.397](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218987379) | [10.796](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218987379) | Samsung S22 |
3230
| MobileNetV2 (mv2) | QNN FP16 | [136.862](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218987097) | [0.673](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218987097) | Samsung S22 |
33-
| InceptionV4 (ic4) | XNN Q8 | [87.617](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218986716) | [117.937](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218986716) | Samsung S22 |
31+
| MobileNetV2 (mv2) | COREML FP16 | [63.019](https://github.com/pytorch/executorch/actions/runs/10892026450/job/30269520397) | [551.031](https://github.com/pytorch/executorch/actions/runs/10892026450/job/30269520397) | Apple iPhone 15 Plus |
32+
| InceptionV4 (ic4) | XNNPACK Q8 | [87.617](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218986716) | [117.937](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218986716) | Samsung S22 |
3433
| InceptionV4 (ic4) | QNN FP16 | [163.943](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218986381) | [2.734](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218986381) | Samsung S22 |
35-
| InceptionV3 (ic3) | XNN Q8 | [60.708](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218986023) | [98.390](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218986023) | Samsung S22 |
34+
| InceptionV4 (ic4) | COREML FP16 | [134.402](https://github.com/pytorch/executorch/actions/runs/10892026450/job/30269518983) | [1906.966](https://github.com/pytorch/executorch/actions/runs/10892026450/job/30269518983) | Apple iPhone 15 Pro |
35+
| InceptionV3 (ic3) | XNNPACK Q8 | [60.708](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218986023) | [98.390](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218986023) | Samsung S22 |
3636
| InceptionV3 (ic3) | QNN FP16 | [134.732](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218985425) | [1.351](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218985425) | Samsung S22 |
37-
| TorchVisionVit (vit) | QNN FP16 | [174.430](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218989581) | [199.279](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218989581) | Samsung S22 |
37+
| InceptionV3 (ic3) | COREML FP16 | [91.808](https://github.com/pytorch/executorch/actions/runs/10892026450/job/30269518511) | [1500.712](https://github.com/pytorch/executorch/actions/runs/10892026450/job/30269518511) | Apple iPhone 15 Pro |
38+
| DeepLabV3 (dl3) | XNNPACK Q8 | [90.616](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218985758) | [666.219](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218985758) | Samsung S22 |
39+
| DeepLabV3 (dl3) | QNN FP16 | [182.207](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218985141) | [9.759](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218985141) | Samsung S22 |
40+
| ResNet50 (resnet50) | XNNPACK Q8 | [55.462](https://github.com/pytorch/executorch/actions/runs/10911725781/job/30285857102) | [37.021](https://github.com/pytorch/executorch/actions/runs/10911725781/job/30285857102) | Apple iPhone 15 Pro |
3841
| ResNet50 (resnet50) | COREML FP16 | [84.453](https://github.com/pytorch/executorch/actions/runs/10836589365/job/30078680731) | [1329.846](https://github.com/pytorch/executorch/actions/runs/10836589365/job/30078680731) | Apple iPhone 15 Pro |
39-
| Wave2Letter (w2l) | XNN Q8 | [33.913](https://github.com/pytorch/executorch/actions/runs/10857890364/job/30139445319) | [135.584](https://github.com/pytorch/executorch/actions/runs/10857890364/job/30139445319) | Apple iPhone 15 Pro |
42+
| TorchVisionVit (vit) | QNN FP16 | [174.430](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218989581) | [199.279](https://github.com/pytorch/executorch/actions/runs/10875550238/job/30218989581) | Samsung S22 |
43+
| Wave2Letter (w2l) | XNNPACK Q8 | [33.913](https://github.com/pytorch/executorch/actions/runs/10857890364/job/30139445319) | [135.584](https://github.com/pytorch/executorch/actions/runs/10857890364/job/30139445319) | Apple iPhone 15 Pro |
4044

4145

4246
## Supported Use Cases

0 commit comments

Comments
 (0)