[executorch][flat tensor] Store number of external tensors in flatbuffer

[lucylq]

Stack from ghstack (oldest at bottom):

• -> [executorch][flat tensor] Store number of external tensors in flatbuffer #8483
• [executorch][flat_tensor] implement load into and dont hold onto the segment #8447
• [flat_tensor] Persist FreeableBuffers of external constants in method #8437

Store the number of external tensors in each ExecutionPlan.

This saves time during loading, as we can directly allocate memory for external tensors without doing a pass to count how many external tensors require allocation.

Differential Revision: D69618283

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8483

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures

As of commit cfd05d6 with merge base c8311e6 ():

NEW FAILURES - The following jobs have failed:

• Check Labels / Check labels (gh)
  RuntimeError: Error checking labels: PR does not have required labels
• Lint / lintrunner / linux-job (gh)
  >>> Lint for backends/cadence/hifi/operators/op_quantized_relu_out.cpp:
• pull / android / run-emulator (gh)
  The process '/usr/bin/sh' failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  exir/_serialize/test/test_program.py::TestProgram::test_unused_inline_delegate_blobs_with_segments
• pull / unittest / macos / macos-job (gh)
  exir/_serialize/test/test_program.py::TestProgram::test_round_trip_large_buffer_sizes

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D69618283

[github-actions]

This PR needs a release notes: label

If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[JacobSzwejbka]

So this is an FC breaking change but thats fine since no one is using external constants yet.

Otherwise adding things to the schema is a pretty high bar typically. I think this change overall is fine since the overhead is small, but I'd still prefer we icebox this change until its clear the overhead of the scan requires this. Since we only ever have to perform the scan if a .ptd was provided anyway.

[larryliu0820]

So this is an FC breaking change but thats fine since no one is using external constants yet.

Otherwise adding things to the schema is a pretty high bar typically. I think this change overall is fine since the overhead is small, but I'd still prefer we icebox this change until its clear the overhead of the scan requires this. Since we only ever have to perform the scan if a .ptd was provided anyway.

Oh just saw this - I would love to make things right before people start to use it.

until its clear the overhead of the scan requires this

Even if the scan is trivial, it is something not necessary at runtime and we probably want the runtime logic to be as simple as possible.

[github-actions]

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

[github-actions]

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.