Write QNN context binary to file when env variable ET_QNN_DEBUG_DIR is set

[limintang]

Differential Revision: D65694854

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6745

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 655b391 with merge base c726a9b ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D65694854

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D65694854

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D65694854

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D65694854

[limintang]

@pytorchbot label "topic: not user facing"

[cccclai]

@chiwwang mind providing some suggestion on this PR?

[chiwwang]

Hi @limintang ,
Could you please share what use-case this PR aim at?
I'm thinking the context binary can be obtained from the .pte file. Or, if we want a pure context-binary without wrapping it into .pte file, we can also write out the file at AOT compilation stage, i.e., add something like fp.write before this line:

executorch/backends/qualcomm/qnn_preprocess.py

Line 113 in 089087b

return PreprocessResult(

The qnn_context_binary should be exactly the QNN context-binary loaded by the runtime.

[limintang]

Hi @limintang , Could you please share what use-case this PR aim at? I'm thinking the context binary can be obtained from the .pte file. Or, if we want a pure context-binary without wrapping it into .pte file, we can also write out the file at AOT compilation stage, i.e., add something like fp.write before this line:

executorch/backends/qualcomm/qnn_preprocess.py

Line 113 in 089087b

return PreprocessResult(

The qnn_context_binary should be exactly the QNN context-binary loaded by the runtime.

We frequently use tools in QNN SDK for various debug and profiling tasks, and these tools consume QNN context binary. We are working with QC to support functionalities provided by these tools programmably so we don't have to rely on tool binaries in the future, but until then, saving the context binary in model export is more convenient.

qnn_preprocess.py is another place where the context binary can be written out, actually I also use this path internally for context binary saving. However, the invocation to this module is from ExecuTorch tracing, I'm not sure whether introducing a platform specific config in the platform agnostic ET module is a good idea.

[chiwwang]

Hi @limintang

Thanks for the response! It makes sense for me.

Just a further question -- do you think environment variable can be a better idea than coupling profiling_level with this need?

I though adding an option to QnnExecuTorchHtpBackendOptions but realized that it's not applicable.
Then I think it would be good if we have runtime option to control this functionality, so environment variable seems a choice.

How about defining an env variable like ET_QNN_BE_DUMP_CNTX_BIN_DIR, if it is not empty, then we dump the context-binaries to the directory it points to?
Is setting env variable possible in your use-case?

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D65694854

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D65694854

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D65694854

[chiwwang]

Looks good for me!

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D65694854

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D65694854

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D65694854

[cccclai]

Sorry a bit late on actually reading this PR...

From reading the conversation, looks like it's mainly to save the qnn_context_binary file, how about we provide a cli tool to unwrap the .pte file to get the qnn_context_binary`, in this case, it can be more generalized. @limintang what do you think?

[limintang]

Sorry a bit late on actually reading this PR...

From reading the conversation, looks like it's mainly to save the qnn_context_binary file, how about we provide a cli tool to unwrap the .pte file to get the qnn_context_binary`, in this case, it can be more generalized. @limintang what do you think?

It would be great if you can provide such tool, then this diff won't be necessary.