Export of internal Abseil changes

--
f49e405201d2ffd5955503fa8ad0f08ec0cdfb2b by Martijn Vels <[email protected]>:

Add common [container.requirements] type definitions to `CharRange` and `ChunkRange`

The presence of these allow these range classes to be used in various utility functions which require some minimum type of container. For example, this change allows tests to use `EXPECT_THAT(cord.Chunks(), ElementsAre(...))`

PiperOrigin-RevId: 406941278

--
0c195f073632e21d9a4bce158047b2ba8551c2d1 by Evan Brown <[email protected]>:

Use explicit exponential growth in SubstituteAndAppendArray.

PiperOrigin-RevId: 406931952

--
afb043bccd809a55cab78abadb7548a057d9eda0 by Jorg Brown <[email protected]>:

Use longer var names in macro to avoid clang-tidy warning

PiperOrigin-RevId: 406930978

--
80397e2604e6b3d929a34742c3a32581b34d3ac4 by Martijn Vels <[email protected]>:

Add future kAppendBuffer and kPrependBuffer API trackers for Cordz sampling

PiperOrigin-RevId: 406912759

--
e910ce919ef83933f08a690e8b7325c7cc5b6d5d by Martijn Vels <[email protected]>:

Implement Prepend(string_view) in terms of PrependArray(string_view, MethodIdentifier).

PiperOrigin-RevId: 406891665

--
c9cff43d4c0568ed01f2fca0f6ef038ae03112b5 by Martijn Vels <[email protected]>:

Add 'Rebuild' logic to CordRepBtree

There are btree hostile scenarios where an application could perform repeated split/insert/merge operations on a cord leading to a tree exceeding the maximum height. While this should be rare in practice, this change adds a Rebuild() method that will rebuild a tree with a 100% fill factor, and we will invoke this rebuild when a tree exceeds the maximum height. This basically follows the similar 'balance' logic in Concat trees (although the latter is common in Concat uses)

PiperOrigin-RevId: 406875739

--
5b2b8fb88f1ebfdc1c670088152da2cb2ea4c376 by Martijn Vels <[email protected]>:

Add 'in place' enabled RemoveSuffix

An in-place RemoveSuffix is more efficient than SubTree() as it can directly modify privately owned nodes and flats allowing easy re-use of free capacity in right-most flats that may turn into Substring edges when using SubTree.

PiperOrigin-RevId: 406431230

--
f09903c0a3d7344f59aaf1380a16ea10829217d4 by Derek Mauro <[email protected]>:

Internal change

PiperOrigin-RevId: 406430373

--
9957af575c33bb18dc170572a4ee8cc5901df6b2 by Greg Falcon <[email protected]>:

Initial groundwork to allow storing checksum data inside CordRep instances.

This uses a RefcountAndFlags bit that was reserved for this purpose, and will be leveraged in a follow-up change to allow attaching checksums to a Cord's value.

This change splits RefcountAndFlags::IsOne() into two distinct operations:

* IsOne(): This returns true when the associated CordRep is not shared with other threads.  This is useful for functions that consume CordRep instances; for example, code that consumes an unshared CordRep can assume ownership of its children without modifying those refcounts.

* IsMutable(): This returns true when the associated CordRep reference is not shared with other threads, *and* does not store an associated checksum value.  This is useful for functions that modify a CordRep's contents: code may modify the bytes of a mutable-unshared CordRep without fear of races with other threads, or of invalidating a stored checksum.

The tricky part of this CL is ensuring that the correct choice between IsMutable() and IsOne() was made at each point.  An incorrect application of IsOne() could lead to correctness bugs in the future.  Code conditioned on IsOne() may delete the CordRep in question, or assume ownership of its children, but must not modify the CordRep's data without explicitly adjusting the CRC.

PiperOrigin-RevId: 406191103

--
686544814079e5ab6d4593cca0c068b510be400a by Martijn Vels <[email protected]>:

Reduce the size in the LargeString test when running with Sanitizers

PiperOrigin-RevId: 406186945

--
735b4490bdb695c35731f06ce4b8de14ce2be6ed by Alex Strelnikov <[email protected]>:

Release absl::SimpleHexAtoi.

PiperOrigin-RevId: 406143188
GitOrigin-RevId: f49e405201d2ffd5955503fa8ad0f08ec0cdfb2b
Change-Id: Ic6527ac40fa03ea02ca813e8bb7868a219544de4

Author: Abseil Team

absl/flags/flag.h

@@ -241,8 +241,8 @@ ABSL_NAMESPACE_END
     /* default value argument. That keeps temporaries alive */               \
     /* long enough for NonConst to work correctly.          */               \
     static constexpr absl::string_view Value(                                \
-        absl::string_view v = ABSL_FLAG_IMPL_FLAGHELP(txt)) {                \
-      return v;                                                              \
+        absl::string_view absl_flag_help = ABSL_FLAG_IMPL_FLAGHELP(txt)) {   \
+      return absl_flag_help;                                                 \
     }                                                                        \
     static std::string NonConst() { return std::string(Value()); }           \
   };                                                                         \
@@ -254,8 +254,8 @@ ABSL_NAMESPACE_END
 #define ABSL_FLAG_IMPL_DECLARE_DEF_VAL_WRAPPER(name, Type, default_value)     \
   struct AbslFlagDefaultGenFor##name {                                        \
     Type value = absl::flags_internal::InitDefaultValue<Type>(default_value); \
-    static void Gen(void* p) {                                                \
-      new (p) Type(AbslFlagDefaultGenFor##name{}.value);                      \
+    static void Gen(void* absl_flag_default_loc) {                            \
+      new (absl_flag_default_loc) Type(AbslFlagDefaultGenFor##name{}.value);  \
     }                                                                         \
   };
 

absl/strings/BUILD.bazel

@@ -305,6 +305,17 @@ cc_library(
     ],
 )
 
+cc_test(
+    name = "cord_internal_test",
+    srcs = ["internal/cord_internal_test.cc"],
+    copts = ABSL_TEST_COPTS,
+    visibility = ["//visibility:private"],
+    deps = [
+        ":cord_internal",
+        "@com_google_googletest//:gtest_main",
+    ],
+)
+
 cc_test(
     name = "cord_rep_btree_test",
     size = "medium",
@@ -684,6 +695,7 @@ cc_test(
         "//absl/base:endian",
         "//absl/base:raw_logging_internal",
         "//absl/container:fixed_array",
+        "//absl/random",
         "@com_google_googletest//:gtest_main",
     ],
 )

absl/strings/CMakeLists.txt

@@ -920,6 +920,7 @@ absl_cc_test(
     absl::cordz_test_helpers
     absl::core_headers
     absl::endian
+    absl::random_random
     absl::raw_logging_internal
     absl::fixed_array
     GTest::gmock_main
@@ -943,6 +944,18 @@ absl_cc_test(
     GTest::gmock_main
 )
 
+absl_cc_test(
+  NAME
+    cord_internal_test
+  SRCS
+    "internal/cord_internal_test.cc"
+  COPTS
+    ${ABSL_TEST_COPTS}
+  DEPS
+    absl::cord_internal
+    GTest::gmock_main
+)
+
 absl_cc_test(
   NAME
     cord_rep_btree_test

absl/strings/cord.cc

@@ -419,7 +419,7 @@ void Cord::InlineRep::PrependTree(CordRep* tree, MethodIdentifier method) {
 // written to region and the actual size increase will be written to size.
 static inline bool PrepareAppendRegion(CordRep* root, char** region,
                                        size_t* size, size_t max_length) {
-  if (root->IsBtree() && root->refcount.IsOne()) {
+  if (root->IsBtree() && root->refcount.IsMutable()) {
     Span<char> span = root->btree()->GetAppendBuffer(max_length);
     if (!span.empty()) {
       *region = span.data();
@@ -430,11 +430,11 @@ static inline bool PrepareAppendRegion(CordRep* root, char** region,
 
   // Search down the right-hand path for a non-full FLAT node.
   CordRep* dst = root;
-  while (dst->IsConcat() && dst->refcount.IsOne()) {
+  while (dst->IsConcat() && dst->refcount.IsMutable()) {
     dst = dst->concat()->right;
   }
 
-  if (!dst->IsFlat() || !dst->refcount.IsOne()) {
+  if (!dst->IsFlat() || !dst->refcount.IsMutable()) {
     *region = nullptr;
     *size = 0;
     return false;
@@ -649,7 +649,7 @@ Cord& Cord::operator=(absl::string_view src) {
   if (tree != nullptr) {
     CordzUpdateScope scope(contents_.cordz_info(), method);
     if (tree->IsFlat() && tree->flat()->Capacity() >= length &&
-        tree->refcount.IsOne()) {
+        tree->refcount.IsMutable()) {
       // Copy in place if the existing FLAT node is reusable.
       memmove(tree->flat()->Data(), data, length);
       tree->length = length;
@@ -819,7 +819,7 @@ void Cord::Prepend(const Cord& src) {
   return Prepend(src_contents);
 }
 
-void Cord::Prepend(absl::string_view src) {
+void Cord::PrependArray(absl::string_view src, MethodIdentifier method) {
   if (src.empty()) return;  // memcpy(_, nullptr, 0) is undefined.
   if (!contents_.is_tree()) {
     size_t cur_size = contents_.inline_size();
@@ -834,7 +834,7 @@ void Cord::Prepend(absl::string_view src) {
     }
   }
   CordRep* rep = NewTree(src.data(), src.size(), 0);
-  contents_.PrependTree(rep, CordzUpdateTracker::kPrependString);
+  contents_.PrependTree(rep, method);
 }
 
 template <typename T, Cord::EnableIfString<T>>
@@ -894,7 +894,7 @@ static CordRep* RemoveSuffixFrom(CordRep* node, size_t n) {
   if (n >= node->length) return nullptr;
   if (n == 0) return CordRep::Ref(node);
   absl::InlinedVector<CordRep*, kInlinedVectorSize> lhs_stack;
-  bool inplace_ok = node->refcount.IsOne();
+  bool inplace_ok = node->refcount.IsMutable();
 
   while (node->IsConcat()) {
     assert(n <= node->length);
@@ -907,7 +907,7 @@ static CordRep* RemoveSuffixFrom(CordRep* node, size_t n) {
       n -= node->concat()->right->length;
       node = node->concat()->left;
     }
-    inplace_ok = inplace_ok && node->refcount.IsOne();
+    inplace_ok = inplace_ok && node->refcount.IsMutable();
   }
   assert(n <= node->length);
 
@@ -968,9 +968,7 @@ void Cord::RemoveSuffix(size_t n) {
     auto constexpr method = CordzUpdateTracker::kRemoveSuffix;
     CordzUpdateScope scope(contents_.cordz_info(), method);
     if (tree->IsBtree()) {
-      CordRep* old = tree;
-      tree = tree->btree()->SubTree(0, tree->length - n);
-      CordRep::Unref(old);
+      tree = CordRepBtree::RemoveSuffix(tree->btree(), n);
     } else {
       CordRep* newrep = RemoveSuffixFrom(tree, n);
       CordRep::Unref(tree);

absl/strings/cord.h

@@ -460,6 +460,16 @@ class Cord {
   // `Cord::chunk_begin()` and `Cord::chunk_end()`.
   class ChunkRange {
    public:
+    // Fulfill minimum c++ container requirements [container.requirements]
+    // Theses (partial) container type definitions allow ChunkRange to be used
+    // in various utilities expecting a subset of [container.requirements].
+    // For example, the below enables using `::testing::ElementsAre(...)`
+    using value_type = absl::string_view;
+    using reference = value_type&;
+    using const_reference = const value_type&;
+    using iterator = ChunkIterator;
+    using const_iterator = ChunkIterator;
+
     explicit ChunkRange(const Cord* cord) : cord_(cord) {}
 
     ChunkIterator begin() const;
@@ -591,6 +601,16 @@ class Cord {
   // `Cord::char_begin()` and `Cord::char_end()`.
   class CharRange {
    public:
+    // Fulfill minimum c++ container requirements [container.requirements]
+    // Theses (partial) container type definitions allow CharRange to be used
+    // in various utilities expecting a subset of [container.requirements].
+    // For example, the below enables using `::testing::ElementsAre(...)`
+    using value_type = char;
+    using reference = value_type&;
+    using const_reference = const value_type&;
+    using iterator = CharIterator;
+    using const_iterator = CharIterator;
+
     explicit CharRange(const Cord* cord) : cord_(cord) {}
 
     CharIterator begin() const;
@@ -881,6 +901,10 @@ class Cord {
   template <typename C>
   void AppendImpl(C&& src);
 
+  // Prepends the provided data to this instance. `method` contains the public
+  // API method for this action which is tracked for Cordz sampling purposes.
+  void PrependArray(absl::string_view src, MethodIdentifier method);
+
   // Assigns the value in 'src' to this instance, 'stealing' its contents.
   // Requires src.length() > kMaxBytesToCopy.
   Cord& AssignLargeString(std::string&& src);
@@ -1220,6 +1244,10 @@ inline void Cord::Append(absl::string_view src) {
   contents_.AppendArray(src, CordzUpdateTracker::kAppendString);
 }
 
+inline void Cord::Prepend(absl::string_view src) {
+  PrependArray(src, CordzUpdateTracker::kPrependString);
+}
+
 extern template void Cord::Append(std::string&& src);
 extern template void Cord::Prepend(std::string&& src);
 

absl/strings/cord_test.cc

@@ -34,6 +34,7 @@
 #include "absl/base/internal/raw_logging.h"
 #include "absl/base/macros.h"
 #include "absl/container/fixed_array.h"
+#include "absl/random/random.h"
 #include "absl/strings/cord_test_helpers.h"
 #include "absl/strings/cordz_test_helpers.h"
 #include "absl/strings/str_cat.h"
@@ -1833,6 +1834,48 @@ TEST_P(CordTest, Hardening) {
   EXPECT_DEATH_IF_SUPPORTED(++cord.chunk_end(), "");
 }
 
+// This test mimics a specific (and rare) application repeatedly splitting a
+// cord, inserting (overwriting) a string value, and composing a new cord from
+// the three pieces. This is hostile towards a Btree implementation: A split of
+// a node at any level is likely to have the right-most edge of the left split,
+// and the left-most edge of the right split shared. For example, splitting a
+// leaf node with 6 edges will result likely in a 1-6, 2-5, 3-4, etc. split,
+// sharing the 'split node'. When recomposing such nodes, we 'injected' an edge
+// in that node. As this happens with some probability on each level of the
+// tree, this will quickly grow the tree until it reaches maximum height.
+TEST_P(CordTest, BtreeHostileSplitInsertJoin) {
+  absl::BitGen bitgen;
+
+  // Start with about 1GB of data
+  std::string data(1 << 10, 'x');
+  absl::Cord buffer(data);
+  absl::Cord cord;
+  for (int i = 0; i < 1000000; ++i) {
+    cord.Append(buffer);
+  }
+
+  for (int j = 0; j < 1000; ++j) {
+    size_t offset = absl::Uniform(bitgen, 0u, cord.size());
+    size_t length = absl::Uniform(bitgen, 100u, data.size());
+    if (cord.size() == offset) {
+      cord.Append(absl::string_view(data.data(), length));
+    } else {
+      absl::Cord suffix;
+      if (offset + length < cord.size()) {
+        suffix = cord;
+        suffix.RemovePrefix(offset + length);
+      }
+      if (cord.size() > offset) {
+        cord.RemoveSuffix(cord.size() - offset);
+      }
+      cord.Append(absl::string_view(data.data(), length));
+      if (!suffix.empty()) {
+        cord.Append(suffix);
+      }
+    }
+  }
+}
+
 class AfterExitCordTester {
  public:
   bool Set(absl::Cord* cord, absl::string_view expected) {

absl/strings/internal/cord_internal.h

@@ -87,6 +87,9 @@ class RefcountAndFlags {
   constexpr RefcountAndFlags() : count_{kRefIncrement} {}
   struct Immortal {};
   explicit constexpr RefcountAndFlags(Immortal) : count_(kImmortalFlag) {}
+  struct WithCrc {};
+  explicit constexpr RefcountAndFlags(WithCrc)
+      : count_(kCrcFlag | kRefIncrement) {}
 
   // Increments the reference count. Imposes no memory ordering.
   inline void Increment() {
@@ -122,14 +125,32 @@ class RefcountAndFlags {
     return count_.load(std::memory_order_acquire) >> kNumFlags;
   }
 
-  // Returns whether the atomic integer is 1.
-  // If the reference count is used in the conventional way, a
-  // reference count of 1 implies that the current thread owns the
-  // reference and no other thread shares it.
-  // This call performs the test for a reference count of one, and
-  // performs the memory barrier needed for the owning thread
-  // to act on the object, knowing that it has exclusive access to the
-  // object.  Always returns false when the immortal bit is set.
+  // Returns true if the referenced object carries a CRC value.
+  bool HasCrc() const {
+    return (count_.load(std::memory_order_relaxed) & kCrcFlag) != 0;
+  }
+
+  // Returns true iff the atomic integer is 1 and this node does not store
+  // a CRC.  When both these conditions are met, the current thread owns
+  // the reference and no other thread shares it, so its contents may be
+  // safely mutated.
+  //
+  // If the referenced item is shared, carries a CRC, or is immortal,
+  // it should not be modified in-place, and this function returns false.
+  //
+  // This call performs the memory barrier needed for the owning thread
+  // to act on the object, so that if it returns true, it may safely
+  // assume exclusive access to the object.
+  inline bool IsMutable() {
+    return (count_.load(std::memory_order_acquire)) == kRefIncrement;
+  }
+
+  // Returns whether the atomic integer is 1.  Similar to IsMutable(),
+  // but does not check for a stored CRC.  (An unshared node with a CRC is not
+  // mutable, because changing its data would invalidate the CRC.)
+  //
+  // When this returns true, there are no other references, and data sinks
+  // may safely adopt the children of the CordRep.
   inline bool IsOne() {
     return (count_.load(std::memory_order_acquire) & kRefcountMask) ==
            kRefIncrement;
@@ -149,14 +170,14 @@ class RefcountAndFlags {
     kNumFlags = 2,
 
     kImmortalFlag = 0x1,
-    kReservedFlag = 0x2,
+    kCrcFlag = 0x2,
     kRefIncrement = (1 << kNumFlags),
 
     // Bitmask to use when checking refcount by equality.  This masks out
     // all flags except kImmortalFlag, which is part of the refcount for
     // purposes of equality.  (A refcount of 0 or 1 does not count as 0 or 1
     // if the immortal bit is set.)
-    kRefcountMask = ~kReservedFlag,
+    kRefcountMask = ~kCrcFlag,
   };
 
   std::atomic<int32_t> count_;