Use content hash instead of UUID for model identifier #16532
Conversation
Summary: Previously, the CoreML backend generated model identifiers using `uuid.uuid4()`, which created a random identifier every time. This meant that even if the exact same model was exported twice, it would get different identifiers, causing cache misses on the device. This change replaces the UUID with a SHA256 hash of the saved mlpackage contents (including weights). Now identical models produce identical identifiers, enabling proper cache hits. The hash is computed by: 1. Saving the model to a temp directory first 2. Hashing all files in the mlpackage directory (sorted for determinism) 3. Using the first 32 characters of the hex digest as the identifier This change also makes it easier to bisect when code changes have altered a generated PTE file - if the identifier changes, you know the model content changed. Note: The identifier format changes from UUID format (with hyphens, e.g., `executorch_a1b2c3d4-e5f6-7890-abcd-ef1234567890`) to a hash format (no hyphens, e.g., `executorch_a1b2c3d4e5f67890abcdef12`). This has no BC impact since the runtime treats the identifier as an opaque string. Differential Revision: D90424166
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/16532
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (1 Unrelated Failure)As of commit 657285f with merge base 0d78c23 ( UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@metascroy has exported this pull request. If you are a Meta employee, you can view the originating Diff in D90424166. |
meta-cla
bot
added
the
CLA Signed
This PR needs a
|
1 participant
Summary:
Previously, the CoreML backend generated model identifiers using
uuid.uuid4(), which created a random identifier every time. This meant that even if the exact same model was exported twice, it would get different identifiers, causing cache misses on the device.This change replaces the UUID with a SHA256 hash of the saved mlpackage contents (including weights). Now identical models produce identical identifiers, enabling proper cache hits.
The hash is computed by:
This change also makes it easier to bisect when code changes have altered a generated PTE file - if the identifier changes, you know the model content changed.
Note: The identifier format changes from UUID format (with hyphens, e.g.,
executorch_a1b2c3d4-e5f6-7890-abcd-ef1234567890) to a hash format (no hyphens, e.g.,executorch_a1b2c3d4e5f67890abcdef12). This has no BC impact since the runtime treats the identifier as an opaque string.Differential Revision: D90424166