[SharedCache] Split state into initial, loaded, and modified

The initial state is initialized during `PerformInitialLoad` and is
immutable after that point. This required some slight restructuring of
how information about memory regions is tracked as that was previously
modified as regions were loaded. Memory regions are now stored in a map
from their address range to the `MemoryRegion` object. This makes it
cheap to look them up by address which is a common operation.

The modified state consists of changes since the last save to the
`DSCView` / `ViewSpecificState`. This means it is no longer necessary to
copy any state when mutating a `SharedCache` instance for the first
time. Instead, its data structures start off empty and are populated as
images, sections, or symbol information is loaded.

The loaded state consists of all modified state that has since been
saved. It lives on the `ViewSpecificState`. Saving modified state
merges it into the the existing loaded state.

This pattern is carried over to the `Metadata` stored on the `DSCView`.
The initial state is stored under its own metadata key, and each
modified state is stored under a key with an incrementing number. This
means each save of the state only needs to serialize the state that
changed, rather than reserializing all of the state all of the time.

There are two huge benefits from these changes:
1. At no point does `SharedCache` have to copy its in memory state.
   The basic copy-on-write approach introduced in #6129 reduced how
   often these copies are made, but they're still frequent and very
   expensive.
1. At no point does `SharedCache` have to re-serialize state to JSON
   that it has already serialized. JSON serialization previously added
   hundreds of milliseconds to any mutating operation on `SharedCache`.

As a result, this speeds up the initial load of the shared cache by
around 2x and loading of subsequent images improves by about the same.

One trade-off is that the serialization / deserialization logic is more
complicated. There are two reasons for this:
1. The state is now split across multiple metadata keys and needs to be
   merged when it is loaded.
2. The in-memory representation uses pointers to identify memory regions.
   These relationships have to be re-established after the JSON is
   deserialized.

As a future direction it is worth considering whether the logic owned by
`SharedCache` could be split in a similar manner to the data. The
initial loading of the cache header, loading of images, and handling of
symbol information are all mostly independent and work on separate data.
If the logic were split into separate classes it would be easier to
reason about which data is valid when, and would easily permit
concurrent loading of multiple images from the shared library in a
thread-safe manner.

Author: bdash

view/sharedcache/api/sharedcache.cpp

@@ -161,6 +161,7 @@ namespace SharedCacheAPI {
 		}
 
 		std::vector<DSCSymbol> result;
+		result.reserve(count);
 		for (size_t i = 0; i < count; i++)
 		{
 			DSCSymbol sym;

view/sharedcache/core/DSCView.cpp

@@ -603,79 +603,21 @@ bool DSCView::Init()
 	Ref<Type> filesetEntryCommandType = Type::StructureType(filesetEntryCommandStruct);
 	DefineType(filesetEntryCommandTypeId, filesetEntryCommandName, filesetEntryCommandType);
 
-	std::vector<SharedCacheCore::MemoryRegion> regionsMappedIntoMemory;
-	if (auto meta = GetParentView()->QueryMetadata(SharedCacheCore::SharedCacheMetadataTag))
+	if (auto metadata = SharedCacheCore::SharedCacheMetadata::LoadFromView(GetParentView()))
 	{
-		std::string data = GetParentView()->GetStringMetadata(SharedCacheCore::SharedCacheMetadataTag);
-		std::stringstream ss;
-		ss.str(data);
-		rapidjson::Document result(rapidjson::kObjectType);
-
-		result.Parse(data.c_str());
-
-		if (result.HasMember("metadataVersion"))
-		{
-			if (result["metadataVersion"].GetInt() != METADATA_VERSION)
-			{
-				LogError("Shared cache metadata version mismatch: expected %d, got %d", METADATA_VERSION,
-					result["metadataVersion"].GetInt());
-				return false;
-			}
-		}
-		else
-		{
-			LogError("Shared cache metadata version not found");
-			return false;
-		}
-		for (auto& imgV : result["regionsMappedIntoMemory"].GetArray())
-		{
-			SharedCacheCore::MemoryRegion region;
-			region.LoadFromValue(imgV);
-			regionsMappedIntoMemory.push_back(region);
-		}
-
-		std::unordered_map<uint64_t, std::string> imageStartToInstallName;
-		// key "m_imageStarts"
-		for (auto& imgV : result["m_imageStarts"].GetArray())
-		{
-			std::string name = imgV.GetArray()[0].GetString();
-			uint64_t addr = imgV.GetArray()[1].GetUint64();
-			imageStartToInstallName[addr] = name;
-		}
-
-		std::vector<std::pair<uint64_t, std::vector<std::pair<uint64_t, std::pair<BNSymbolType, std::string>>>>> exportInfos;
-
-		for (const auto& obj1 : result["exportInfos"].GetArray())
-		{
-			std::vector<std::pair<uint64_t, std::pair<BNSymbolType, std::string>>> innerVec;
-			for (const auto& obj2 : obj1["value"].GetArray())
-			{
-				std::pair<BNSymbolType, std::string> innerPair = { (BNSymbolType)obj2["val1"].GetUint64(), obj2["val2"].GetString() };
-				innerVec.push_back({ obj2["key"].GetUint64(), innerPair });
-			}
-
-			exportInfos.push_back({obj1["key"].GetUint64(), innerVec});
-		}
-
 		BeginBulkModifySymbols();
-		for (const auto & [imageBaseAddr, exportList] : exportInfos)
+		for (const auto& [imageBaseAddr, exportMap] : metadata->ExportInfos())
 		{
-			std::vector<Ref<Symbol>> symbolsList;
-			for (const auto & [exportAddr, exportTypeAndName] : exportList)
-			{
-				symbolsList.push_back(new Symbol(exportTypeAndName.first, exportTypeAndName.second, exportAddr));
-			}
-
-			auto typelib = GetTypeLibrary(imageStartToInstallName[imageBaseAddr]);
+			auto typelib = GetTypeLibrary(metadata->InstallNameForImageBaseAddress(imageBaseAddr));
 
-			for (const auto& symbol : symbolsList)
+			for (const auto& [address, symbol] : *exportMap)
 			{
-				if (!IsValidOffset(symbol->GetAddress()))
+				if (!IsValidOffset(address))
 					continue;
+
 				if (typelib)
 				{
-					auto type = typelib->GetNamedObject(symbol->GetRawName());
-					if (type)
+					if (auto type = typelib->GetNamedObject(symbol->GetFullName()))
 					{
 						DefineAutoSymbolAndVariableOrFunction(GetDefaultPlatform(), symbol, type);
 						continue;

view/sharedcache/core/MetadataSerializable.cpp

@@ -47,6 +47,11 @@ void Serialize(SerializationContext& context, uint64_t value) {
 	context.writer.Uint64(value);
 }
 
+void Serialize(SerializationContext& context, unsigned long value)
+{
+	context.writer.Uint64(value);
+}
+
 void Deserialize(DeserializationContext& context, std::string_view name, bool& b) {
 	b = context.doc[name.data()].GetBool();
 }

view/sharedcache/core/MetadataSerializable.hpp

@@ -86,9 +86,10 @@ struct DeserializationContext {
 template <typename Derived, typename LoadResult = Derived>
 class MetadataSerializable {
 public:
-	std::string AsString() const {
+	template <typename... Args>
+	std::string AsString(Args&&... args) const {
 		SerializationContext context;
-		Store(context);
+		Store(context, std::forward<Args>(args)...);
 
 		return context.buffer.GetString();
 	}
@@ -106,14 +107,15 @@ class MetadataSerializable {
 		return Derived::Load(context);
 	}
 
-	Ref<Metadata> AsMetadata() const {
-		return new Metadata(AsString());
+	template <typename... Args>
+	Ref<Metadata> AsMetadata(Args&&... args) const {
+		return new Metadata(AsString(std::forward<Args>(args)...));
 	}
 
 	template <typename... Args>
-	void Store(SerializationContext& context) const {
+	void Store(SerializationContext& context, Args&&... args) const {
 		context.writer.StartObject();
-		AsDerived().Store(context);
+		AsDerived().Store(context, std::forward<Args>(args)...);
 		context.writer.EndObject();
 	}
 
@@ -148,8 +150,8 @@ void Serialize(SerializationContext& context, const std::pair<First, Second>& va
 	context.writer.EndArray();
 }
 
-template <typename K, typename V>
-void Serialize(SerializationContext& context, const std::map<K, V>& value)
+template <typename K, typename V, typename L>
+void Serialize(SerializationContext& context, const std::map<K, V, L>& value)
 {
 	context.writer.StartArray();
 	for (auto& pair : value)
@@ -181,6 +183,15 @@ void Serialize(SerializationContext& context, const std::vector<T>& values)
 	context.writer.EndArray();
 }
 
+template <typename T>
+void Serialize(SerializationContext& context, const std::optional<T>& value)
+{
+	if (value.has_value())
+		Serialize(context, *value);
+	else
+		context.writer.Null();
+}
+
 SHAREDCACHE_FFI_API void Serialize(SerializationContext& context, const char*);
 SHAREDCACHE_FFI_API void Serialize(SerializationContext& context, bool b);
 SHAREDCACHE_FFI_API void Deserialize(DeserializationContext& context, std::string_view name, bool& b);
@@ -200,6 +211,7 @@ SHAREDCACHE_FFI_API void Serialize(SerializationContext& context, int32_t b);
 SHAREDCACHE_FFI_API void Deserialize(DeserializationContext& context, std::string_view name, int32_t& b);
 SHAREDCACHE_FFI_API void Serialize(SerializationContext& context, int64_t b);
 SHAREDCACHE_FFI_API void Deserialize(DeserializationContext& context, std::string_view name, int64_t& b);
+SHAREDCACHE_FFI_API void Serialize(SerializationContext& context, unsigned long b);
 SHAREDCACHE_FFI_API void Serialize(SerializationContext& context, std::string_view b);
 SHAREDCACHE_FFI_API void Serialize(SerializationContext& context, const std::pair<uint64_t, std::pair<uint64_t, uint64_t>>& value);
 SHAREDCACHE_FFI_API void Deserialize(DeserializationContext& context, std::string_view name, std::string& b);