Write QNN context binary to file when env variable ET_QNN_DEBUG_DIR is set #6745

limintang · 2024-11-09T03:53:58Z

Differential Revision: D65694854

pytorch-bot · 2024-11-09T03:54:02Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6745

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 655b391 with merge base c726a9b ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-11-09T03:54:14Z

This pull request was exported from Phabricator. Differential Revision: D65694854

…torch#6745) Summary: Pull Request resolved: pytorch#6745 Differential Revision: D65694854

facebook-github-bot · 2024-11-09T03:59:57Z

This pull request was exported from Phabricator. Differential Revision: D65694854

…torch#6745) Summary: Pull Request resolved: pytorch#6745 Differential Revision: D65694854

facebook-github-bot · 2024-11-13T23:07:59Z

This pull request was exported from Phabricator. Differential Revision: D65694854

…torch#6745) Summary: Pull Request resolved: pytorch#6745 Reviewed By: billmguo Differential Revision: D65694854

facebook-github-bot · 2024-11-18T19:51:21Z

This pull request was exported from Phabricator. Differential Revision: D65694854

limintang · 2024-11-21T04:14:34Z

@pytorchbot label "topic: not user facing"

cccclai · 2024-11-22T21:45:06Z

@chiwwang mind providing some suggestion on this PR?

chiwwang · 2024-11-25T05:13:33Z

Hi @limintang ,
Could you please share what use-case this PR aim at?
I'm thinking the context binary can be obtained from the .pte file. Or, if we want a pure context-binary without wrapping it into .pte file, we can also write out the file at AOT compilation stage, i.e., add something like fp.write before this line:

executorch/backends/qualcomm/qnn_preprocess.py

Line 113 in 089087b

return PreprocessResult(

The qnn_context_binary should be exactly the QNN context-binary loaded by the runtime.

limintang · 2024-11-26T22:16:20Z

Hi @limintang , Could you please share what use-case this PR aim at? I'm thinking the context binary can be obtained from the .pte file. Or, if we want a pure context-binary without wrapping it into .pte file, we can also write out the file at AOT compilation stage, i.e., add something like fp.write before this line:

executorch/backends/qualcomm/qnn_preprocess.py

Line 113 in 089087b

return PreprocessResult(

The qnn_context_binary should be exactly the QNN context-binary loaded by the runtime.

We frequently use tools in QNN SDK for various debug and profiling tasks, and these tools consume QNN context binary. We are working with QC to support functionalities provided by these tools programmably so we don't have to rely on tool binaries in the future, but until then, saving the context binary in model export is more convenient.

qnn_preprocess.py is another place where the context binary can be written out, actually I also use this path internally for context binary saving. However, the invocation to this module is from ExecuTorch tracing, I'm not sure whether introducing a platform specific config in the platform agnostic ET module is a good idea.

chiwwang

Hi @limintang

Thanks for the response! It makes sense for me.

Just a further question -- do you think environment variable can be a better idea than coupling profiling_level with this need?

I though adding an option to QnnExecuTorchHtpBackendOptions but realized that it's not applicable.
Then I think it would be good if we have runtime option to control this functionality, so environment variable seems a choice.

How about defining an env variable like ET_QNN_BE_DUMP_CNTX_BIN_DIR, if it is not empty, then we dump the context-binaries to the directory it points to?
Is setting env variable possible in your use-case?

backends/qualcomm/runtime/QnnManager.cpp

…torch#6745) Summary: Pull Request resolved: pytorch#6745 Reviewed By: billmguo Differential Revision: D65694854

facebook-github-bot · 2024-11-27T07:16:43Z

This pull request was exported from Phabricator. Differential Revision: D65694854

…torch#6745) Summary: Pull Request resolved: pytorch#6745 Reviewed By: billmguo Differential Revision: D65694854

facebook-github-bot · 2024-11-27T07:17:47Z

This pull request was exported from Phabricator. Differential Revision: D65694854

…s set (pytorch#6745) Summary: Pull Request resolved: pytorch#6745 Reviewed By: billmguo Differential Revision: D65694854

facebook-github-bot · 2024-11-27T07:21:46Z

This pull request was exported from Phabricator. Differential Revision: D65694854

chiwwang

Looks good for me!

backends/qualcomm/runtime/QnnManager.cpp

…s set (pytorch#6745) Summary: Pull Request resolved: pytorch#6745 Reviewed By: billmguo Differential Revision: D65694854

facebook-github-bot · 2024-11-27T09:40:44Z

This pull request was exported from Phabricator. Differential Revision: D65694854

…s set (pytorch#6745) Summary: Pull Request resolved: pytorch#6745 Reviewed By: billmguo Differential Revision: D65694854

facebook-github-bot · 2024-11-27T09:44:59Z

This pull request was exported from Phabricator. Differential Revision: D65694854

…s set (pytorch#6745) Summary: Pull Request resolved: pytorch#6745 Reviewed By: billmguo Differential Revision: D65694854

facebook-github-bot · 2024-11-27T23:38:37Z

This pull request was exported from Phabricator. Differential Revision: D65694854

cccclai · 2024-11-29T04:31:24Z

Sorry a bit late on actually reading this PR...

From reading the conversation, looks like it's mainly to save the qnn_context_binary file, how about we provide a cli tool to unwrap the .pte file to get the qnn_context_binary`, in this case, it can be more generalized. @limintang what do you think?

limintang · 2024-12-02T17:31:35Z

Sorry a bit late on actually reading this PR...

From reading the conversation, looks like it's mainly to save the qnn_context_binary file, how about we provide a cli tool to unwrap the .pte file to get the qnn_context_binary`, in this case, it can be more generalized. @limintang what do you think?

It would be great if you can provide such tool, then this diff won't be necessary.

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 9, 2024

facebook-github-bot added the fb-exported label Nov 9, 2024

limintang added a commit to limintang/executorch that referenced this pull request Nov 9, 2024

Write QNN context binary to file when profiling level is detailed (py…

15cfbfc

…torch#6745) Summary: Pull Request resolved: pytorch#6745 Differential Revision: D65694854

limintang force-pushed the export-D65694854 branch from b8dfa7d to 15cfbfc Compare November 9, 2024 03:59

limintang force-pushed the export-D65694854 branch from 15cfbfc to 71eecb5 Compare November 13, 2024 23:07

limintang added a commit to limintang/executorch that referenced this pull request Nov 13, 2024

Write QNN context binary to file when profiling level is detailed (py…

71eecb5

…torch#6745) Summary: Pull Request resolved: pytorch#6745 Differential Revision: D65694854

cccclai requested review from chiwwang, chunit-quic, haowhsu-quic and shewu-quic November 13, 2024 23:19

limintang force-pushed the export-D65694854 branch from 71eecb5 to 63299b7 Compare November 18, 2024 19:51

limintang added a commit to limintang/executorch that referenced this pull request Nov 18, 2024

Write QNN context binary to file when profiling level is detailed (py…

63299b7

…torch#6745) Summary: Pull Request resolved: pytorch#6745 Reviewed By: billmguo Differential Revision: D65694854

pytorch-bot bot added the topic: not user facing label Nov 21, 2024

chiwwang reviewed Nov 27, 2024

View reviewed changes

backends/qualcomm/runtime/QnnManager.cpp Outdated Show resolved Hide resolved

limintang force-pushed the export-D65694854 branch from 73acc22 to f5c21ee Compare November 27, 2024 07:16

limintang added a commit to limintang/executorch that referenced this pull request Nov 27, 2024

Write QNN context binary to file when profiling level is detailed (py…

f5c21ee

…torch#6745) Summary: Pull Request resolved: pytorch#6745 Reviewed By: billmguo Differential Revision: D65694854

limintang added a commit to limintang/executorch that referenced this pull request Nov 27, 2024

Write QNN context binary to file when profiling level is detailed (py…

0849df3

…torch#6745) Summary: Pull Request resolved: pytorch#6745 Reviewed By: billmguo Differential Revision: D65694854

limintang force-pushed the export-D65694854 branch from f5c21ee to 0849df3 Compare November 27, 2024 07:17

limintang force-pushed the export-D65694854 branch from 0849df3 to a65d435 Compare November 27, 2024 07:21

limintang changed the title ~~Write QNN context binary to file when profiling level is detailed~~ Write QNN context binary to file when env variable ET_QNN_DEBUG_DIR is set Nov 27, 2024

chiwwang approved these changes Nov 27, 2024

View reviewed changes

shewu-quic reviewed Nov 27, 2024

View reviewed changes

backends/qualcomm/runtime/QnnManager.cpp Outdated Show resolved Hide resolved

limintang force-pushed the export-D65694854 branch from a65d435 to e10ebd1 Compare November 27, 2024 09:40

limintang force-pushed the export-D65694854 branch from e10ebd1 to 4bb5be4 Compare November 27, 2024 09:44

limintang requested a review from shewu-quic November 27, 2024 19:43

Write QNN context binary to file when env variable ET_QNN_DEBUG_DIR i…

655b391

…s set (pytorch#6745) Summary: Pull Request resolved: pytorch#6745 Reviewed By: billmguo Differential Revision: D65694854

limintang force-pushed the export-D65694854 branch from 4bb5be4 to 655b391 Compare November 27, 2024 23:38

limintang closed this Dec 31, 2024

limintang deleted the export-D65694854 branch December 31, 2024 06:05

haowhsu-quic mentioned this pull request Jan 24, 2025

Qualcomm AI Engine Direct - context dump utility #7931

Merged

Write QNN context binary to file when env variable ET_QNN_DEBUG_DIR is set #6745

Write QNN context binary to file when env variable ET_QNN_DEBUG_DIR is set #6745

Uh oh!

Conversation

limintang commented Nov 9, 2024

Uh oh!

pytorch-bot bot commented Nov 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6745

✅ No Failures

Uh oh!

facebook-github-bot commented Nov 9, 2024

Uh oh!

facebook-github-bot commented Nov 9, 2024

Uh oh!

facebook-github-bot commented Nov 13, 2024

Uh oh!

facebook-github-bot commented Nov 18, 2024

Uh oh!

limintang commented Nov 21, 2024

Uh oh!

cccclai commented Nov 22, 2024

Uh oh!

chiwwang commented Nov 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

limintang commented Nov 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chiwwang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

facebook-github-bot commented Nov 27, 2024

Uh oh!

facebook-github-bot commented Nov 27, 2024

Uh oh!

facebook-github-bot commented Nov 27, 2024

Uh oh!

chiwwang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

facebook-github-bot commented Nov 27, 2024

Uh oh!

facebook-github-bot commented Nov 27, 2024

Uh oh!

facebook-github-bot commented Nov 27, 2024

Uh oh!

cccclai commented Nov 29, 2024

Uh oh!

limintang commented Dec 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

pytorch-bot bot commented Nov 9, 2024 •

edited

Loading

chiwwang commented Nov 25, 2024 •

edited

Loading

limintang commented Nov 26, 2024 •

edited

Loading