-
Notifications
You must be signed in to change notification settings - Fork 108
Ln/replace best chain with block index #1413
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
134 commits
Select commit
Hold shift + click to select a range
fc9a0d3
start by refactoring block index into a read-through cache
lazynina f3e8e4c
best chain and header chain with read through cache
lazynina a2e5b4b
check bestHeaderChain.ChainMap ONLY when processing pos headers
lazynina ff6ee24
updates for locating headers and faster cache usage
lazynina 71b1eaa
more fixes
lazynina 17a56bd
more fixes
lazynina d153927
more fix, specifically around .Ancestor
lazynina 47e1958
bump test timeout
lazynina 5e5c0c1
load all blocks after committed tip in block index
lazynina 9a32c26
fix GetBlockNodeByHashAndHeight - no need for checking height > maxHe…
lazynina 5359be8
bump timeout to 60m
lazynina 89a845e
try hashicorp lru cache
lazynina 4f7ff12
Fix up some more tests and remove logging
lazynina 722dcc0
fix string slice workaround, fix calc next difficulty
lazynina 1941487
use best header chain instead of best chain for calc next difficulty …
lazynina 627e882
fix tests
lazynina a469c3a
refactor to just use a block hash for best block chain and best heade…
lazynina c6ad24a
move block index migration to NewBlockchain instead of NewServer, add…
lazynina 216fd31
merge main into branch
lazynina 1f80f19
Merge branch 'ln/read-through-cache-for-block-index-and-best-chain' i…
lazynina 8b73b1c
handle case where we haven't hit PoS cutover yet in block index migra…
lazynina 7ac67bd
cleanup
lazynina 7675765
do some sampling for is fully stored and check archival mode
lazynina 148bad3
fix get blocks to store
lazynina 8aa6a5b
cleanup
lazynina 53b2d29
make a bunch of methods on block index private and expose block index…
lazynina 91e22ff
don't bother healing orphan pointers if we're syncing
lazynina 9abe238
refactor to reduce number of times we have to call hash and don't bot…
lazynina 23991c4
upgrade cloudflare/circl
lazynina 8adc624
Ln/replace best chain plus replace old lru cache (#1414)
lazynina 2608ec5
Merge branch 'ln/replace-best-chain-with-block-index' of github.com:d…
lazynina dbad649
go mod tidy
lazynina ebb1175
expose SetBestChain
lazynina ae34cff
fix exit condition for GetBlocksToStore
lazynina b48c52d
use bytes.Equal instead of reflect.DeepEqual for comparing block hashes
lazynina 925e44e
use get block from best chain by height to speed up syncing
lazynina dddd64c
fix condition for returning from get block by hash
lazynina 62503c1
revert using get block from best chain by height
lazynina 131f389
add height check to reduce calls to compare hashes
lazynina f57a699
fix exit condition for height in get block from best chain by hash
lazynina 2bae1a2
allow early return from GetBlockFromBestChainByHash if we're using he…
lazynina ea243f3
minor enhancement to isTipCurrent
lazynina 3f73bb6
fix txindex issue w/ genesis block
lazynina e200407
fix process block pow logic for genesis block
lazynina 3ad68be
fix - get next block after tip hash in update process of txindex
lazynina 78034d4
reduce direct access to BlockNode.Parent and get rid of heal orphan p…
lazynina 02e9e15
fix postgres get best chain logic
lazynina dec4675
merge in main
lazynina 2a82dea
Ln/downgrade badger (#1430)
lazynina d7b6f93
resolve conflicts with main
lazynina 453cf51
resolve merge conflicts
lazynina 35f7a6a
remove LocateBestBlockChainHeaders and replace with DH suggestion. fl…
lazynina 69e1825
make GetUtxoViewAndUtxoOpsAtBlockHash public
lazynina a859c72
cleanup
lazynina 01cd500
change GetBlockFromBestChainByHash to GetBlockFromBestChainByHashAndO…
lazynina 09091df
use GetBlockFromBestChainByHashAndOptionalHeight in GetBlockNodesToFetch
lazynina 77a1f57
nuke locator and inventory logic, just return a slice of the tip hash
lazynina 0585ebc
move check in should verify signatures up in function to reduce amoun…
lazynina 139fb2c
reduce size of block index cache
lazynina 4259c39
use header chain for get blocks to store
lazynina aea51a8
fix return value from processHeaderPoW
lazynina fc63a94
disable deadlock detection around putting block nodes into db after h…
lazynina 90b5372
add more logging as we're processing headers after finishing download…
lazynina f9a4f3e
iterate backwards to update status of block nodes after finishing hyp…
lazynina f425317
fix set tip call since we reverse backwards through the block nodes a…
lazynina 231a6dc
small clean up, add some todos
lazynina ca70c8a
add more comments and TODOs and clean up for all files except blockch…
lazynina 05a6041
Merge branch 'main' into ln/replace-best-chain-with-block-index
lazynina a9b549a
contains -> exists for lru cache
lazynina 4caabaa
add comments in blockchain.go
lazynina 8a0bf0f
add more comments and cleanup. remove block index by height, just go …
lazynina 7b311fd
Fix tests and always flush PoW headers in process header pow so calc …
lazynina 01abf3d
add comment about slowness
lazynina 091ad57
write faster GetHeadersForLocatorAndStopHash
lazynina 9ed19df
fix typo
lazynina 969cb1c
include committed tip in header locator, fix walkback to committed ti…
lazynina a8384cb
adjust log level for highest committed block in GetUtxoViewandUtxoOps…
lazynina 8aa423e
clean up comments
lazynina e6aeff6
remove load block index from height function. it's not necessary
lazynina 8ad8155
fix TestShouldReorg
lazynina 8656873
revert change to should reorg
lazynina 5f00b26
only compute BMF if we're passed PoS block height
lazynina c58b2a1
remove usage of ParsePubKey from btcec lib in connect block logic
lazynina f727ec4
adjust log level for get utxo view and utxo ops at block hash. add lo…
lazynina 80f3bf7
add enhancement to get cached block view
lazynina 754b0a6
remove unnecessary log
lazynina 65dc6a6
fix block index migration
lazynina 760a5f0
don't use cache in get utxo view and utxo ops at block hash
lazynina f6d8c1c
use GetParent where applicable. fix start up
lazynina 55cc676
add comment about why 'optimization' actually doesn't work
lazynina bb09433
Revert one usage of GetParent
lazynina e115f1c
log start of handleBlockProposalEvent, simplify get safe block nodes
lazynina 4975854
put getMaxSequentialBlockHeightAfter back in
lazynina 2f3724d
don't need to call getMaxSequentialBlockHeightAfter
lazynina 2ba0c66
add param to enumerate keys for prefix to disable prefetch
lazynina 11aedb6
try using stream for iteration
lazynina d2475da
revert usage of stream api in GetBlockNodesByHeight
lazynina 57a43a7
reduce logging in pos_consensus.go
lazynina 652e4e2
add more logging
lazynina 2e83071
add more logging as suggested by DH
lazynina cd0c78e
add logging in getStoredLineageFromCommittedTip
lazynina 3ac6ba2
add one more log
lazynina 87c35e4
add more logging in getStoredLineage
lazynina 0c8606c
Ln/replace best chain plus no blocknode parent (#1439)
lazynina 844a28e
Adjust log levels and only handle inv requests when chain state is fu…
lazynina d130837
reduce log level, fix PG tests
lazynina 36b605e
go mod tidy
lazynina 0a9499e
reduce size of walkback in initChain to 6 hours
lazynina 11773dd
update GetSpendableDeSoBalanceNanosForPublicKey
lazynina 587ac72
fix GetSpendableUtxosForPublicKey
lazynina 05a2ae3
revert changes for get spendable balance and utxos. update get spenda…
lazynina d40b2f1
Merge branch 'main' into ln/replace-best-chain-with-block-index
lazynina f963570
update some error messages
lazynina 8d8bb6d
switch HandleInv to respond based on isSyncing function rather than c…
lazynina db8469b
skipping invs for peer - log level 3
lazynina f2bf0d7
loop over locators in GetHeadersForLocatorAndStopHash
lazynina ca638f9
switch handleInv to only when fully current
lazynina ea85d4e
skip update after connect/disconnect block hooks in legacy mempool i…
lazynina e73d998
update GetBlockNodesToFetch to use GetParent
lazynina ca1d91b
Clean up
lazynina 301891f
revert HandleInv to use isSyncing instead of checking for SyncStateFu…
lazynina 9f25d89
Merge branch 'main' into ln/replace-best-chain-with-block-index
lazynina 0654f4c
Merge branch 'main' into ln/replace-best-chain-with-block-index
lazynina 04375be
fix IsFullyStored
lazynina d0428c2
Revert LatestHeaderLocator changes
lazynina d748ca6
more closely replicate old LatestLocator function, also use this func…
lazynina bc87a14
use DerivedKeyEntry.Copy in utxo view copy
lazynina dfcdb14
add optimization in get block nodes to fetch
lazynina 65e6244
Merge branch 'main' into ln/replace-best-chain-with-block-index
lazynina 0ac4385
Merge branch 'main' into ln/replace-best-chain-with-block-index
lazynina 837ea3b
resolve merge conflicts
lazynina e07c942
add validatorPK in error about duplicate validator public key
lazynina 38029eb
use true for isHeaderChain in LatestLocator
lazynina 66ddf70
reverse slice at the end of GetHeadersForLocatorAndStopHash instead o…
lazynina File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| package collections | ||
|
|
||
| import lru "github.com/hashicorp/golang-lru/v2" | ||
|
|
||
| // We implement our own LRU cache as a wrapper around a dependency. This allows us to | ||
| // easily change the underlying implementation in the future if needed with minimal | ||
| // changes outside of this file. | ||
|
|
||
| type LruCache[K comparable, V any] struct { | ||
| underlyingCache *lru.Cache[K, V] | ||
| } | ||
|
|
||
| func NewLruCache[K comparable, V any](maxSize int) (*LruCache[K, V], error) { | ||
| underlyingCache, err := lru.New[K, V](maxSize) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| return &LruCache[K, V]{underlyingCache}, nil | ||
| } | ||
|
|
||
| func (lruCache *LruCache[K, V]) Put(key K, value V) { | ||
| lruCache.underlyingCache.Add(key, value) | ||
| } | ||
|
|
||
| func (lruCache *LruCache[K, V]) Get(key K) (V, bool) { | ||
| return lruCache.underlyingCache.Get(key) | ||
| } | ||
|
|
||
| func (lruCache *LruCache[K, V]) Exists(key K) bool { | ||
| return lruCache.underlyingCache.Contains(key) | ||
| } | ||
|
|
||
| func (lruCache *LruCache[K, V]) Delete(key K) { | ||
| lruCache.underlyingCache.Remove(key) | ||
| } | ||
|
|
||
| func (lruCache *LruCache[K, V]) Purge() { | ||
| lruCache.underlyingCache.Purge() | ||
| } | ||
|
|
||
| func (lruCache *LruCache[K, V]) Keys() []K { | ||
| return lruCache.underlyingCache.Keys() | ||
| } | ||
|
|
||
| type LruSet[K comparable] struct { | ||
| underlyingCache *lru.Cache[K, struct{}] | ||
| } | ||
|
|
||
| func NewLruSet[K comparable](maxSize int) (*LruSet[K], error) { | ||
| underlyingCache, err := lru.New[K, struct{}](maxSize) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| return &LruSet[K]{underlyingCache}, nil | ||
| } | ||
|
|
||
| func (lruSet *LruSet[K]) Put(key K) { | ||
| lruSet.underlyingCache.Add(key, struct{}{}) | ||
| } | ||
|
|
||
| func (lruSet *LruSet[K]) Contains(key K) bool { | ||
| return lruSet.underlyingCache.Contains(key) | ||
| } | ||
|
|
||
| func (lruSet *LruSet[K]) Delete(key K) { | ||
| lruSet.underlyingCache.Remove(key) | ||
| } | ||
|
|
||
| func (lruSet *LruSet[K]) Purge() { | ||
| lruSet.underlyingCache.Purge() | ||
| } | ||
|
|
||
| func (lruSet *LruSet[K]) Items() []K { | ||
| return lruSet.underlyingCache.Keys() | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.