Skip to content

Commit 4fac767

Browse files
committed
gc: replace ordered sets with unordered sets for in-memory caches
During garbage collection we cache several things -- a set of known-dead paths, a set of known-alive paths, and a map of paths to their derivers. Currently they use STL maps and sets, which are ordered structures that typically are backed by binary trees. Since we are putting pseudorandom paths into these and looking them up by exact key, we don't need the ordering, and we're paying a nontrivial cost per insertion. The existing maps require O(n log n) memory and have O(log n) insertion and lookup time. We could instead use unordered maps, which are typically backed by hashmaps. These require O(n) memory and have O(1) insertion and lookup time. On my system this appears to result in a dramatic speedup -- prior to this patch I was able to delete 400k paths out of 9.5 million over the course of 34.5 hours. After this patch the same result took 89 minutes. This result should NOT be taken at face value because the two runs aren't really comparable; in particular the first started when I had 9.5 million store paths and the seconcd started with 7.8 million, so we are deleting a different set of paths starting from a much cleaner filesystem. But I do think it's indicative. Related: #9581
1 parent a44ae8b commit 4fac767

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

src/libstore/gc.cc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -455,7 +455,7 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
455455
bool gcKeepOutputs = settings.gcKeepOutputs;
456456
bool gcKeepDerivations = settings.gcKeepDerivations;
457457

458-
StorePathSet roots, dead, alive;
458+
std::unordered_set<StorePath> roots, dead, alive;
459459

460460
struct Shared
461461
{
@@ -661,7 +661,7 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
661661
}
662662
};
663663

664-
std::map<StorePath, StorePathSet> referrersCache;
664+
std::unordered_map<StorePath, StorePathSet> referrersCache;
665665

666666
/* Helper function that visits all paths reachable from `start`
667667
via the referrers edges and optionally derivers and derivation

0 commit comments

Comments
 (0)