Deserialize with named data store #14743
Conversation
As titled
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14743
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New Failure, 1 Unrelated FailureAs of commit 5edbffd with merge base f24351a ( NEW FAILURE - The following job has failed:
BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
This PR needs a
|
| ) | ||
| # Keep a convenient mapping from key to raw bytes for callers that only | ||
| # need to read the blobs. | ||
| setattr( # noqa: B010 |
There was a problem hiding this comment.
Think we don't need this if we have the named_data_store in line 683.
Not sure about attaching named_data_store to program type though?
| if isinstance(value, str): | ||
| data[key] = T[value] | ||
| else: | ||
| data[key] = T(value) |
| {key: buffers[idx].buffer for key, idx in pte_named_data.items()}, | ||
| ) | ||
| self.assertTrue(hasattr(program2, "named_data_blobs")) | ||
| self.assertEqual(program2.named_data_blobs, restored_named_data) |
There was a problem hiding this comment.
Test the roundtrip --> can re-serialize the program with program2.named_data_blobs?
This pull request enhances the deserialization and restoration logic for named data segments in programs, ensuring that named data is reconstructed into a dedicated store and verified in tests. It also includes minor code quality improvements.
Named Data Segment Restoration:
_restore_segments(exir/_serialize/_program.py), added logic to reconstruct named data blobs into aNamedDataStoreOutputand attach bothnamed_data_storeand a convenientnamed_data_blobsmapping to theProgramobject. The originalnamed_datalist is cleared after restoration.test_constant_delegate_and_named_data_segmentsandtest_named_data_segmentsinexir/_serialize/test/test_program.py) to verify that named data is correctly restored, removed from theProgram, and accessible via the new store and blob mapping. [1] [2]