Arm backend: Add dump_delegate_data function

[wwwind]

Add dump_delegate_data function

Signed-off-by: Elena Zhelezina [email protected]

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12334

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 10 New Failures, 2 Cancelled Jobs, 1 Unrelated Failure

As of commit 74b59af with merge base bf4f0a3 ():

NEW FAILURES - The following jobs have failed:

• Build documentation / build (buck2) / Build doc (gh)
  At least one of the pre-conditions you specified did not hold
• Lint / lintrunner / linux-job (gh)
  >>> Lint for exir/program/_program.py:
• pull / unittest / linux / linux-job (gh)
  examples/models/llama/tests/test_ring_attention.py::TestRingAttention::test_sliding_window_attention_custom
• pull / unittest-editable / linux / linux-job (gh)
  examples/models/llama/tests/test_ring_attention.py::TestRingAttention::test_sliding_window_attention_custom
• trunk / test-huggingface-transformers-coreml (olmo-1b|coreml_fp32_gpu|--quantize) / macos-job (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• trunk / test-huggingface-transformers-coreml (qwen3-1.7b|xnnpack|--quantize) / macos-job (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• trunk / test-models-arm-zephyr (add) / linux-job (gh)
  RuntimeError: Command docker exec -t 35e71070175ea61e410466294a7c981482bfc03109910ab05ed385870939596a /exec failed with exit code 1
• trunk / test-models-arm-zephyr (mv2) / linux-job (gh)
  RuntimeError: Command docker exec -t a5ae0660ae1db776ee3690ca8f19b18c3564cdfec908e5eafeb960b71cb6a63f /exec failed with exit code 1
• trunk / test-models-arm-zephyr (softmax) / linux-job (gh)
  RuntimeError: Command docker exec -t a3464be38f8cd701feedeabf05cfb112b45dc4e88119e256c6a172acd925b4a0 /exec failed with exit code 1
• trunk / unittest-release / linux / linux-job (gh)
  examples/models/llama/tests/test_ring_attention.py::TestRingAttention::test_sliding_window_attention_custom

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• pull / test-llama-runner-linux-android / linux-job (gh)
• pull / test-models-linux (resnet18, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• trunk / unittest-release / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 8

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[JacobSzwejbka]

can you take a TextIO instead? Like Dump_et_program above

edit: Or just any stream really since the data isnt text

[JacobSzwejbka]

Can you not find the blob if its embedded in the flatbuffer section?

[digantdesai]

Thanks @wwwind for the PR.

Is the goal for extracting the blobs is to run a delegate subgraph on a TOSA reference model? If yes, then I don't think I prefer this approach of extracting the blobs and running them.

The main reason is e2e validation. You can have N blobs which will lead to N tosa_ref_sim calls and then you might have to stitch them together manually to see if the entire network worked correctly or not.

Alternatively I propose we can write a tosa runtime which works with tosa partitioner and graph breaks can be handled by portable libs seamlessly. This way you can re-use all the ET infra and have e2e working TOSA PTE :)

[wwwind]

Thank you for the review @digantdesai

Actually, this function is for cases when we don't need e2e flow. We have a graphic use case when we get these blobs as .tosa files and then we have a plugin which implements different bits around it in shaders and pass some data to run subgraphs.

[digantdesai]

Thank you for the review @digantdesai

Actually, this function is for cases when we don't need e2e flow. We have a graphic use case when we get these blobs as .tosa files and then we have a plugin which implements different bits around it in shaders and pass some data to run subgraphs.

Ok. Curious why can't we dump them after generation from preprocess? Extracting feels a bit like going backwards.

[wwwind]

I leave this to @per to answer as I am not sure how this could be done after preprocess

[per]

Ok. Curious why can't we dump them after generation from preprocess? Extracting feels a bit like going backwards.

Agree, it is a bit like going backwards, but with a dedicated API intstead of using different flags for the backend to dump the delegate data. For the Arm backends it will be usuable for gettning the delegate data out of the .pte for all our backends, while the primary driver is the VgfBackend and internal testing scenarios.
Gold would be to also have some dump method (akin to objdump) on the serialized .pte files (maybe a executorch.runtime.Program method) to be able to extract the delegate data for a program serialized to file.

[digantdesai]

Ok. Curious why can't we dump them after generation from preprocess? Extracting feels a bit like going backwards.

Agree, it is a bit like going backwards, but with a dedicated API intstead of using different flags for the backend to dump the delegate data. For the Arm backends it will be usuable for gettning the delegate data out of the .pte for all our backends, while the primary driver is the VgfBackend and internal testing scenarios. Gold would be to also have some dump method (akin to objdump) on the serialized .pte files (maybe a executorch.runtime.Program method) to be able to extract the delegate data for a program serialized to file.

Ok I don't have strong opinions, and don't want to block this. @JacobSzwejbka would be more appropriate for the review.

That said, thinking out loud why VGFPartitioner(dump_blobs=True) wouldn't be a better interface? Let me think some more about your comment for delegate agnostic AoT API.

The reason I am hesitant with the current approach is that (1) it doesn't let you avoid the AoT PTE generation flow, hence the opinion about the AoT API, (2) pte_objdump is being considered however that is more FC/BC tooling maintenance around the PTE as a file format, esp if we are expecting users to go through AoT flow anyway. And will get more complicated with NamedDataMap etc. features to share tensors across delegates.

[digantdesai]

Any update on this?

[wwwind]

@digantdesai @per What is the final decision on this API ? Do we go forward with it or should I abandon this PR ?

[github-actions]

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.