[CAS] Update CAS implementation according to upstream version

Update downstream code to match upstreamed version.

Author: cachemeifyoucan

llvm/docs/ContentAddressableStorage.md

@@ -0,0 +1,121 @@
+# Content Addressable Storage
+
+## Introduction to CAS
+
+Content Addressable Storage, or `CAS`, is a storage system that assigns
+unique addresses to the data stored. It is very useful for data deduplicaton
+and creating unique identifiers.
+
+Unlike other kinds of storage systems, like file systems, CAS is immutable. It
+is more reliable to model a computation by representing the inputs and outputs
+of the computation using objects stored in CAS.
+
+The basic unit of the CAS library is a CASObject, where it contains:
+
+* Data: arbitrary data
+* References: references to other CASObject
+
+It can be conceptually modeled as something like:
+
+```
+struct CASObject {
+  ArrayRef<char> Data;
+  ArrayRef<CASObject*> Refs;
+}
+```
+
+With this abstraction, it is possible to compose `CASObject`s into a DAG that is
+capable of representing complicated data structures, while still allowing data
+deduplication. Note you can compare two DAGs by just comparing the CASObject
+hash of two root nodes.
+
+
+## LLVM CAS Library User Guide
+
+The CAS-like storage provided in LLVM is `llvm::cas::ObjectStore`.
+To reference a CASObject, there are few different abstractions provided
+with different trade-offs:
+
+### ObjectRef
+
+`ObjectRef` is a lightweight reference to a CASObject stored in the CAS.
+This is the most commonly used abstraction and it is cheap to copy/pass
+along. It has following properties:
+
+* `ObjectRef` is only meaningful within the `ObjectStore` that created the ref.
+`ObjectRef` created by different `ObjectStore` cannot be cross-referenced or
+compared.
+* `ObjectRef` doesn't guarantee the existence of the CASObject it points to. An
+explicit load is required before accessing the data stored in CASObject.
+This load can also fail, for reasons like (but not limited to): object does
+not exist, corrupted CAS storage, operation timeout, etc.
+* If two `ObjectRef` are equal, it is guaranteed that the object they point to
+are identical (if they exist). If they are not equal, the underlying objects are
+guaranteed to be not the same.
+
+### ObjectProxy
+
+`ObjectProxy` represents a loaded CASObject. With an `ObjectProxy`, the
+underlying stored data and references can be accessed without the need
+of error handling. The class APIs also provide convenient methods to
+access underlying data. The lifetime of the underlying data is equal to
+the lifetime of the instance of `ObjectStore` unless explicitly copied.
+
+### CASID
+
+`CASID` is the hash identifier for CASObjects. It owns the underlying
+storage for hash value so it can be expensive to copy and compare depending
+on the hash algorithm. `CASID` is generally only useful in rare situations
+like printing raw hash value or exchanging hash values between different
+CAS instances with the same hashing schema.
+
+### ObjectStore
+
+`ObjectStore` is the CAS-like object storage. It provides API to save
+and load CASObjects, for example:
+
+```
+ObjectRef A, B, C;
+Expected<ObjectRef> Stored = ObjectStore.store("data", {A, B});
+Expected<ObjectProxy> Loaded = ObjectStore.getProxy(C);
+```
+
+It also provides APIs to convert between `ObjectRef`, `ObjectProxy` and
+`CASID`.
+
+
+
+## CAS Library Implementation Guide
+
+The LLVM ObjectStore API was designed so that it is easy to add
+customized CAS implementations that are interchangeable with the builtin
+ones.
+
+To add your own implementation, you just need to add a subclass to
+`llvm::cas::ObjectStore` and implement all its pure virtual methods.
+To be interchangeable with LLVM ObjectStore, the new CAS implementation
+needs to conform to following contracts:
+
+* Different CASObjects stored in the ObjectStore need to have a different hash
+and result in a different `ObjectRef`. Similarly, the same CASObject should have
+the same hash and the same `ObjectRef`. Note: two different CASObjects with
+identical data but different references are considered different objects.
+* `ObjectRef`s are only comparable within the same `ObjectStore` instance, and
+can be used to determine the equality of the underlying CASObjects.
+* The loaded objects from the ObjectStore need to have a lifetime at least as
+long as the ObjectStore itself so it is always legal to access the loaded data
+without holding on the `ObjectProxy` until the `ObjectStore` is destroyed.
+
+
+If not specified, the behavior can be implementation defined. For example,
+`ObjectRef` can be used to point to a loaded CASObject so
+`ObjectStore` never fails to load. It is also legal to use a stricter model
+than required. For example, the underlying value inside `ObjectRef` can be
+the unique indentities of the objects across multiple `ObjectStore` instances,
+but comparing such `ObjectRef` from different `ObjectStore` is still illegal.
+
+For CAS library implementers, there is also an `ObjectHandle` class that
+is an internal representation of a loaded CASObject reference.
+`ObjectProxy` is just a pair of `ObjectHandle` and `ObjectStore`, and
+just like `ObjectRef`, `ObjectHandle` is only useful when paired with
+the `ObjectStore` that knows about the loaded CASObject.

llvm/docs/Reference.rst

@@ -17,6 +17,7 @@ LLVM and API reference documentation.
    CalleeTypeMetadata
    CIBestPractices
    CommandGuide/index
+   ContentAddressableStorage
    ConvergenceAndUniformity
    ConvergentOperations
    Coroutines
@@ -244,3 +245,6 @@ Additional Topics
 :doc:`MLGO`
    Facilities for ML-Guided Optimization, such as collecting IR corpora from a
    build, interfacing with ML models, an exposing features for training.
+
+:doc:`ContentAddressableStorage`
+   A reference guide for using LLVM's CAS library.

llvm/include/llvm/CAS/BuiltinCASContext.h

@@ -54,6 +54,7 @@ namespace llvm::cas::builtin {
 using HasherT = BLAKE3;
 using HashType = decltype(HasherT::hash(std::declval<ArrayRef<uint8_t> &>()));
 
+/// CASContext for LLVM builtin CAS using BLAKE3 hash type.
 class BuiltinCASContext : public CASContext {
   void printIDImpl(raw_ostream &OS, const CASID &ID) const final;
   void anchor() override;

llvm/include/llvm/CAS/BuiltinObjectHasher.h

@@ -9,13 +9,12 @@
 #ifndef LLVM_CAS_BUILTINOBJECTHASHER_H
 #define LLVM_CAS_BUILTINOBJECTHASHER_H
 
-#include "llvm/ADT/StringRef.h"
 #include "llvm/CAS/ObjectStore.h"
 #include "llvm/Support/Endian.h"
 
-namespace llvm {
-namespace cas {
+namespace llvm::cas {
 
+/// Hasher for stored objects in builtin CAS.
 template <class HasherT> class BuiltinObjectHasher {
 public:
   using HashT = decltype(HasherT::hash(std::declval<ArrayRef<uint8_t> &>()));
@@ -64,21 +63,20 @@ template <class HasherT> class BuiltinObjectHasher {
 
   void updateArray(ArrayRef<char> Bytes) {
     updateArray(ArrayRef(reinterpret_cast<const uint8_t *>(Bytes.data()),
-                             Bytes.size()));
+                         Bytes.size()));
   }
 
   void updateSize(uint64_t Size) {
     Size = support::endian::byte_swap(Size, endianness::little);
-    Hasher.update(ArrayRef(reinterpret_cast<const uint8_t *>(&Size),
-                                sizeof(Size)));
+    Hasher.update(
+        ArrayRef(reinterpret_cast<const uint8_t *>(&Size), sizeof(Size)));
   }
 
   BuiltinObjectHasher() = default;
   ~BuiltinObjectHasher() = default;
   HasherT Hasher;
 };
 
-} // namespace cas
-} // namespace llvm
+} // namespace llvm::cas
 
 #endif // LLVM_CAS_BUILTINOBJECTHASHER_H

llvm/include/llvm/CAS/CASID.h

@@ -55,18 +55,21 @@ class CASContext {
 /// compared directly. If they are, then \a
 /// CASIDContext::getHashSchemaIdentifier() is compared to see if they can be
 /// compared by hash, in which case the result of \a getHash() is compared.
-///
-/// FIXME: Rename to ObjectID (and rename file to CASObjectID.h?).
 class CASID {
 public:
   void dump() const;
-  void print(raw_ostream &OS) const {
-    return getContext().printIDImpl(OS, *this);
-  }
+
   friend raw_ostream &operator<<(raw_ostream &OS, const CASID &ID) {
     ID.print(OS);
     return OS;
   }
+
+  /// Print CASID.
+  void print(raw_ostream &OS) const {
+    return getContext().printIDImpl(OS, *this);
+  }
+
+  /// Return a printable string for CASID.
   std::string toString() const;
 
   ArrayRef<uint8_t> getHash() const {
@@ -110,6 +113,7 @@ class CASID {
 
   CASID() = delete;
 
+  /// Create CASID from CASContext and raw hash bytes.
   static CASID create(const CASContext *Context, StringRef Hash) {
     return CASID(Context, Hash);
   }

llvm/include/llvm/CAS/CASReference.h

@@ -20,7 +20,6 @@ class raw_ostream;
 namespace cas {
 
 class ObjectStore;
-
 class ObjectHandle;
 class ObjectRef;
 
@@ -41,8 +40,9 @@ class ReferenceBase {
     return InternalRef;
   }
 
+  /// Helper functions for DenseMapInfo.
   unsigned getDenseMapHash() const {
-    return (unsigned)llvm::hash_value(InternalRef);
+    return static_cast<unsigned>(llvm::hash_value(InternalRef));
   }
   bool isDenseMapEmpty() const { return InternalRef == getDenseMapEmptyRef(); }
   bool isDenseMapTombstone() const {
@@ -89,7 +89,7 @@ class ReferenceBase {
 #endif
 };
 
-/// Reference to an object in a \a ObjectStore instance.
+/// Reference to an object in an \a ObjectStore instance.
 ///
 /// If you have an ObjectRef, you know the object exists, and you can point at
 /// it from new nodes with \a ObjectStore::store(), but you don't know anything
@@ -105,12 +105,6 @@ class ReferenceBase {
 /// ObjectHandle, a variant that knows what kind of entity it is. \a
 /// ObjectStore::getReferenceKind() can expect the type of reference without
 /// asking for unloaded objects to be loaded.
-///
-/// This is a wrapper around a \c uint64_t (and a \a ObjectStore instance when
-/// assertions are on). If necessary, it can be deconstructed and reconstructed
-/// using \a Reference::getInternalRef() and \a
-/// Reference::getFromInternalRef(), but clients aren't expected to need to do
-/// this. These both require the right \a ObjectStore instance.
 class ObjectRef : public ReferenceBase {
   struct DenseMapTag {};
 

llvm/include/llvm/CAS/FileSystemCache.h

@@ -14,7 +14,6 @@
 #include "llvm/ADT/ScopeExit.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/CAS/CASReference.h"
-#include "llvm/CAS/HashMappedTrie.h"
 #include "llvm/CAS/ThreadSafeAllocator.h"
 #include "llvm/Support/AlignOf.h"
 #include "llvm/Support/Allocator.h"