Peras staging ground #1581
Draft
Peras staging ground #1581
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
amesgen
requested review from
nfrisby,
jasagredo,
fraser-iohk,
dnadales and
geo2a
as code owners
1 task
2 tasks
amesgen
force-pushed
the
peras-staging
branch
3 times, most recently
from
4c69d1c to
1390449
Compare
It is not necessary to perform the garbage collection of the LedgerDB and the map of invalid blocks in the same STM transaction. In the past, this was important, but it is not anymore, see #1507.
Primarily, this is an optimization to reduce the maximum memory usage (more relevant with the in-memory backend) when pruning happens on garbage collection instead of while adding new blocks to the LedgerDB, see the added commit and the benchmark in the pull request. Previously, LedgerDB garbage collection happened as part of VolatileDB garbage collection, which was intentionally rate-limited. Also, it resolves the current (somewhat weird) behavior that we do not copy any blocks to the ImmutableDB when we are taking a snapshot (which can take >2 minutes), and consequently also not garbage-collecting the VolatileDB. It also synergizes with the planned feature to add a random delay when taking snapshots.
Also make sure to account for the fact that the DbChangelog might have gotten pruned between opening and committing the forker.
regarding the previous few commits
It was already superseded in the most important places due to `LedgerDbPruneBeforeSlot`. Its remaining use cases are non-essential: - Replay on startup. In this case, we never roll back, so not maintaining k states is actually an optimization here. We can also remove the now-redundant `InitDB.pruneDb` function. - Internal functions used for db-analyser. Here, we can just as well use `LedgerDbPruneAll` (which is used by `pruneToImmTipOnly`) as we never need to roll back. - Testing. In particular, we remove some DbChangelog tests that previously ensured that only at most @k@ states are kept. This is now no longer true; that property is instead enforced by the LedgerDB built on top of the DbChangelog. A follow-up commit in this PR enriches the LedgerDB state machine test to make sure that the public API functions behave appropriately, ensuring that we don't lose test coverage (and also testing V2, which previously didn't have any such tests).
Make sure that we correctly fail when trying to roll back too far.
using cabal-docspec (not yet nixified)
This is purely for concise QuickCheck counterexample output.
This is in preparation for weighted chain comparisons.
Also remove the version for `ValidatedChainDiff` as it is unused.
…-diffusion-testlib
… associated smoke tests
…n), and the associated smoke test
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Peras staging ground, only used for running CI