Implement python deserialize for flat_tensor #11779
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/11779
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 7a33ea8 with merge base 5960a4b ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
|
|
||
| self.assertEqual(segments[2].offset + segments[2].size, len(segment_data)) | ||
|
|
||
| def test_round_trip(self) -> None: |
There was a problem hiding this comment.
Note to reviewers: I considered adding a deserialize test in the same vein as the serialize test, but it's a bit awkward to do, in that we'd need to construct a binary input without using the existing serialization logic, Could also include a pre-serialized artifact, but I think this coverage is sufficient in conjunction with the serialize test. Feel free to let me know if you want to see more coverage.
GregoryComer
added
the
release notes: none
GregoryComer
force-pushed
the
flat-tensor-deserialize
branch
from
1422bfe to
7a33ea8
Compare
lucylq
left a comment
lucylq
left a comment
There was a problem hiding this comment.
LGTM, thank you for adding this!
| deserialized_payload = serializer.deserialize(Cord(serialized_data)) | ||
|
|
||
| # Validate the deserialized payload. Since alignment isn't serialized, we need to | ||
| # do this somewhat manually. |
There was a problem hiding this comment.
I don't think there's currently a need to serialize the alignment(?)
It might be good to have for deserialization/general error-checking, I guess the trade-off is binary size.
| if field.name not in SKIP_FIELDS: | ||
| self.assertEqual( | ||
| getattr(reference, field.name), | ||
| getattr(actual, field.name), |
### Summary Implement the deserialize function in the Python FlatTensorSerializer class. This allows for loading flatbuffer data from a Python environment. ### Test plan I've added an additional test in `extension/flat_tensor/test/test_serialize.py` to cover the deserialize path.
3 participants
Summary
Implement the deserialize function in the Python FlatTensorSerializer class. This allows for loading flatbuffer data from a Python environment.
Test plan
I've added an additional test in
extension/flat_tensor/test/test_serialize.pyto cover the deserialize path.