[XNNPACK][Weights Cache] Use sha256 hash of bytes instead of tensor name (pytorch#9413)

In production use cases, I've become increasingly afraid of the Weights Cache managing weights across multiple models and the potential for collisions on names. Names like "encoder.layer.weight1" are popular names for encoder models, and that name may be reused across many different models. In reality such a tensor found in different models will be different.

A way to alleviate such concerns around collisions is to provide a strong hashing guarantee around the tensor's bytes. Namely if we use the sha256 hash of the tensor bytes as the named key we would have much stronger guarantees around the potential of collisions between weights.

Additionally this can provide stronger weight deduplication guarantees. For now we use the named key as the only method for deduplicating weights, but if the underlying bytes are the same but the keys are different we won't be able to deduplicate. Using a hash on the underlying bytes as a key though would help with this (though how likely this happens remains to be seen). Regardless i think hashing the bytes will be much safer in the long-term.

The draw back is that this adds a guaranteed 64 bytes per weight. On smaller models this might amount to a bit. Open to discuss on whether other hashing algorithms might provide tolerable collision guarantees like: md5_hash.

Differential Revision: [D71212509](https://our.internmc.facebook.com/intern/diff/D71212509/)

Co-authored-by: Max Ren <[email protected]>

Author: pytorchbot

backends/xnnpack/operators/node_visitor.py

@@ -5,6 +5,7 @@
 # LICENSE file in the root directory of this source tree.
 
 import ctypes
+import hashlib
 
 from typing import cast, Dict, List, Optional, Tuple
 
@@ -34,7 +35,6 @@
     check_or_raise,
     get_input_node,
     get_param_tensor,
-    get_tensor_name,
     is_param_node,
     PERM_NCHW_TO_NHWC,
 )
@@ -576,15 +576,19 @@ def get_serialized_buffer_index(
         if quant_params is not None and quant_params.is_qc4w:
             const_val = self.convert_to_qc4w(const_val)
 
-        array_type = ctypes.c_char * const_val.untyped_storage().nbytes()
+        size = const_val.untyped_storage().nbytes()
+        array_type = ctypes.c_char * size
         array = ctypes.cast(
             const_val.untyped_storage().data_ptr(),
             ctypes.POINTER(array_type),
         ).contents
 
-        named_key = get_tensor_name(self.exported_program, get_attr_node)
-        if named_key == "":
-            raise ValueError(f"Tensor from node: {get_attr_node} has no name")
+        check_or_raise(
+            size > 0,
+            f"Serializing constant data node {tensor} but tensor value has no bytes",
+        )
+        sha256_hash = hashlib.sha256(bytes(array))
+        named_key = sha256_hash.hexdigest()
 
         size = const_val.untyped_storage().nbytes()
         xnn_graph.constant_data.append(