[llvm][cas] New mapped file design to fix filesystem assumptions

On the advice of filesystem experts, we are changing our approach to
the memory-mapped files used in the CAS. Specifically, we will no longer
attempt to resize the underlying file while holding an mmap, which does
not behave correctly on all filesystems (e.g. tmpfs)  and is formally
unspecified by POSIX. Ideally, we could simply update the mapped regions
whenever we resize, but in practice this is challenging to do portably,
cross-process, without locking the reading process' access, and without
requiring us to check every offset before reading it and trigger a
re-map if necessary (which is hard to test/maintain).

Instead, we will truncate the file to its maximum size immediately, and
resize it back to the allocated size when closing, if possible. On
filesystems with sparse file support, this does not require any
additional storage. See the comments in MappedFileRegionBumpPtr for
details.

Because the test suite is uniquely likley to (a) crash from time to
time, and (b) have many separate CAS instances for tests, we override
the usual index/data/actioncache file sizes to be at most 100 MB. This
should help mitigate the risks of having many large CAS instances left
around if testing on a filesystem without sparse file support.

rdar://105905301
(cherry picked from commit 0046c5d)

Author: benlangmuir

clang/test/lit.cfg.py

@@ -297,3 +297,9 @@ def exclude_unsupported_files_for_aix(dirname):
 # possibly be present in system and user configuration files, so disable
 # default configs for the test runs.
 config.environment["CLANG_NO_DEFAULT_CONFIG"] = "1"
+
+# Restrict the size of the on-disk CAS for tests. This allows testing in
+# constrained environments (e.g. small TMPDIR). It also prevents leaving
+# behind large files on file systems that do not support sparse files if a test
+# crashes before resizing the file.
+config.environment["LLVM_CAS_MAX_MAPPING_SIZE"] = "%d" % (100 * 1024 * 1024)

llvm/include/llvm/CAS/LazyMappedFileRegion.h

@@ -0,0 +1,126 @@
+//===- MappedFileRegionBumpPtr.h --------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CAS_MAPPEDFILEREGIONBUMPPTR_H
+#define LLVM_CAS_MAPPEDFILEREGIONBUMPPTR_H
+
+#include "llvm/Config/llvm-config.h"
+#include "llvm/Support/Alignment.h"
+#include "llvm/Support/FileSystem.h"
+#include <atomic>
+
+namespace llvm::cas {
+
+/// Allocator for an owned mapped file region that supports thread-safe and
+/// process-safe bump pointer allocation.
+///
+/// This allocator is designed to create a sparse file when supported by the
+/// filesystem's \c ftruncate so that it can be used with a large maximum size.
+/// It will also attempt to shrink the underlying file down to its current
+/// allocation size when the last concurrent mapping is closed.
+///
+/// Process-safe. Uses file locks when resizing the file during initialization
+/// and destruction.
+///
+/// Thread-safe, assuming all threads use the same instance to talk to a given
+/// file/mapping. Unsafe to have multiple instances talking to the same file
+/// in the same process since file locks will misbehave. Clients should
+/// coordinate (somehow).
+///
+/// \note Currently we allocate the whole file without sparseness on Windows.
+///
+/// Provides 8-byte alignment for all allocations.
+class MappedFileRegionBumpPtr {
+public:
+  using RegionT = sys::fs::mapped_file_region;
+
+  /// Create a \c MappedFileRegionBumpPtr.
+  ///
+  /// \param Path the path to open the mapped region.
+  /// \param Capacity the maximum size for the mapped file region.
+  /// \param BumpPtrOffset the offset at which to store the bump pointer.
+  /// \param NewFileConstructor is for constructing new files. It has exclusive
+  /// access to the file. Must call \c initializeBumpPtr.
+  static Expected<MappedFileRegionBumpPtr>
+  create(const Twine &Path, uint64_t Capacity, int64_t BumpPtrOffset,
+         function_ref<Error(MappedFileRegionBumpPtr &)> NewFileConstructor);
+
+  /// Create a \c MappedFileRegionBumpPtr., shared across the process via a
+  /// singleton map.
+  ///
+  /// FIXME: Singleton map should be based on sys::fs::UniqueID, but currently
+  /// it is just based on \p Path.
+  ///
+  /// \param Path the path to open the mapped region.
+  /// \param Capacity the maximum size for the mapped file region.
+  /// \param BumpPtrOffset the offset at which to store the bump pointer.
+  /// \param NewFileConstructor is for constructing new files. It has exclusive
+  /// access to the file. Must call \c initializeBumpPtr.
+  static Expected<std::shared_ptr<MappedFileRegionBumpPtr>> createShared(
+      const Twine &Path, uint64_t Capacity, int64_t BumpPtrOffset,
+      function_ref<Error(MappedFileRegionBumpPtr &)> NewFileConstructor);
+
+  /// Finish initializing the bump pointer. Must be called by
+  /// \c NewFileConstructor.
+  void initializeBumpPtr(int64_t BumpPtrOffset);
+
+  /// Minimum alignment for allocations, currently hardcoded to 8B.
+  static constexpr Align getAlign() {
+    // Trick Align into giving us '8' as a constexpr.
+    struct alignas(8) T {};
+    static_assert(alignof(T) == 8, "Tautology failed?");
+    return Align::Of<T>();
+  }
+
+  /// Allocate at least \p AllocSize. Rounds up to \a getAlign().
+  char *allocate(uint64_t AllocSize) {
+    return data() + allocateOffset(AllocSize);
+  }
+  /// Allocate, returning the offset from \a data() instead of a pointer.
+  int64_t allocateOffset(uint64_t AllocSize);
+
+  char *data() const { return Region.data(); }
+  uint64_t size() const { return *BumpPtr; }
+  uint64_t capacity() const { return Region.size(); }
+
+  RegionT &getRegion() { return Region; }
+
+  ~MappedFileRegionBumpPtr() { destroyImpl(); }
+
+  MappedFileRegionBumpPtr() = default;
+  MappedFileRegionBumpPtr(MappedFileRegionBumpPtr &&RHS) { moveImpl(RHS); }
+  MappedFileRegionBumpPtr &operator=(MappedFileRegionBumpPtr &&RHS) {
+    destroyImpl();
+    moveImpl(RHS);
+    return *this;
+  }
+
+  MappedFileRegionBumpPtr(const MappedFileRegionBumpPtr &) = delete;
+  MappedFileRegionBumpPtr &operator=(const MappedFileRegionBumpPtr &) = delete;
+
+private:
+  void destroyImpl();
+  void moveImpl(MappedFileRegionBumpPtr &RHS) {
+    std::swap(Region, RHS.Region);
+    std::swap(BumpPtr, RHS.BumpPtr);
+    std::swap(Path, RHS.Path);
+    std::swap(FD, RHS.FD);
+    std::swap(SharedLockFD, RHS.SharedLockFD);
+  }
+
+private:
+  RegionT Region;
+  std::atomic<int64_t> *BumpPtr = nullptr;
+  std::string Path;
+  std::optional<int> FD;
+  std::optional<int> SharedLockFD;
+};
+
+} // namespace llvm::cas
+
+#endif // LLVM_CAS_MAPPEDFILEREGIONBUMPPTR_H

llvm/include/llvm/CAS/LazyMappedFileRegionBumpPtr.h

@@ -48,8 +48,9 @@ class FileOffset {
 ///   memory mapping. Atomic access does not work on networked filesystems.
 /// - Filesystem locks are used, but only sparingly:
 ///     - during initialization, for creating / opening an existing store;
-///     - rarely on insertion, to resize the store (see \a
-///       OnDiskHashMappedTrie::MappedFileInfo::requestFileSize()).
+///     - for the lifetime of the instance, a shared/reader lock is held
+///     - during destruction, if there are no concurrent readers, to shrink the
+///       files to their minimum size.
 /// - Path is used as a directory:
 ///     - "index" stores the root trie and subtries.
 ///     - "data" stores (most of) the entries, like a bump-ptr-allocator.

llvm/include/llvm/CAS/MappedFileRegionBumpPtr.h

@@ -16,10 +16,10 @@ add_llvm_component_library(LLVMCAS
   HashMappedTrie.cpp
   HierarchicalTreeBuilder.cpp
   InMemoryCAS.cpp
-  LazyMappedFileRegion.cpp
-  LazyMappedFileRegionBumpPtr.cpp
+  MappedFileRegionBumpPtr.cpp
   ObjectStore.cpp
   OnDiskCAS.cpp
+  OnDiskCommon.cpp
   OnDiskGraphDB.cpp
   OnDiskHashMappedTrie.cpp
   OnDiskKeyValueDB.cpp