[pull] master from bnb-chain:master#118
Merged
pull[bot] merged 453 commits intodysnix:masterfrom Aug 28, 2025
Merged
Conversation
TruncatePending shows up bright red on our nodes, because it computes
the length of a map multiple times.
I don't know why this is so expensive, but around 20% of our time is
spent on this, which is super weird.
```
//PR: BenchmarkTruncatePending-24 17498 69397 ns/op 32872 B/op 3 allocs/op
//Master: BenchmarkTruncatePending-24 9960 123954 ns/op 32872 B/op 3 allocs/op
```
```
benchmark old ns/op new ns/op delta
BenchmarkTruncatePending-24 123954 69397 -44.01%
benchmark old allocs new allocs delta
BenchmarkTruncatePending-24 3 3 +0.00%
benchmark old bytes new bytes delta
BenchmarkTruncatePending-24 32872 32872 +0.00%
```
This simple PR is a 44% improvement over the old state
```
OUTINE ======================== github.com/ethereum/go-ethereum/core/txpool/legacypool.(*LegacyPool).truncatePending in github.com/ethereum/go-ethereum/core/txpool/legacypool/legacypool.go
1.96s 18.02s (flat, cum) 19.57% of Total
. . 1495:func (pool *LegacyPool) truncatePending() {
. . 1496: pending := uint64(0)
60ms 2.99s 1497: for _, list := range pool.pending {
250ms 5.48s 1498: pending += uint64(list.Len())
. . 1499: }
. . 1500: if pending <= pool.config.GlobalSlots {
. . 1501: return
. . 1502: }
. . 1503:
. . 1504: pendingBeforeCap := pending
. . 1505: // Assemble a spam order to penalize large transactors first
. 510ms 1506: spammers := prque.New[int64, common.Address](nil)
140ms 2.50s 1507: for addr, list := range pool.pending {
. . 1508: // Only evict transactions from high rollers
50ms 5.08s 1509: if uint64(list.Len()) > pool.config.AccountSlots {
. . 1510: spammers.Push(addr, int64(list.Len()))
. . 1511: }
. . 1512: }
. . 1513: // Gradually drop transactions from offenders
. . 1514: offenders := []common.Address{}
```
```go
// Benchmarks the speed of batch transaction insertion in case of multiple accounts.
func BenchmarkTruncatePending(b *testing.B) {
// Generate a batch of transactions to enqueue into the pool
pool, _ := setupPool()
defer pool.Close()
b.ReportAllocs()
batches := make(types.Transactions, 4096+1024+1)
for i := range len(batches) {
key, _ := crypto.GenerateKey()
account := crypto.PubkeyToAddress(key.PublicKey)
pool.currentState.AddBalance(account, uint256.NewInt(1000000), tracing.BalanceChangeUnspecified)
tx := transaction(uint64(0), 100000, key)
batches[i] = tx
}
for _, tx := range batches {
pool.addRemotesSync([]*types.Transaction{tx})
}
b.ResetTimer()
// benchmark truncating the pending
for range b.N {
pool.truncatePending()
}
}
```
This PR adds checking for an edgecase which theoretically can happen in the range-prover. Right now, we check that a key does not overwrite a previous one by checking that the key is increasing. However, if keys are of different lengths, it is possible to create a key which is increasing _and_ overwrites the previous key. Example: `0xaabbcc` followed by `0xaabbccdd`. This can not happen in go-ethereum, which always uses fixed-size paths for accounts and storage slot paths in the trie, but it might happen if the range prover is used without guaranteed fixed-size keys. This PR also adds some testcases for the errors that are expected.
This PR applies the config overrides to the new config as well, otherwise they will not be applied to defined configs, making shadowforks impossible. To test: ``` > ./build/bin/geth --override.prague 123 --dev --datadir /tmp/geth INFO [04-28|21:20:47.009] - Prague: @123 > ./build/bin/geth --override.prague 321 --dev --datadir /tmp/geth INFO [04-28|21:23:59.760] - Prague: @321 ``
For PeerDAS, we need to compute cell proofs. Both ckzg and gokzg support computing these cell proofs. This PR does the following: - Update the go-kzg library from "github.com/crate-crypto/go-kzg-4844" to "github.com/crate-crypto/go-eth-kzg" which will be the new upstream for go-kzg moving forward - Update ckzg from v1.0.0 to v2.0.1 and switch to /v2 - Updates the trusted setup to contain the g1 points both in lagrange and monomial form - Expose `ComputeCells` to compute the cell proofs
This changes the filtermaps to only pull up the raw receipts, not the derived receipts which saves a lot of allocations. During normal execution this will reduce the allocations of the whole geth node by ~15%.
Since the block hash is not returned for pending blocks, ethclient cannot unmarshal into RPC block. This makes hash optional on rpc block and compute the hash locally for pending blocks to correctly key the tx sender cache. https://github.com/ethereum/go-ethereum/blob/a82303f4e3cedcebe31540a53dde4f24fc93da80/internal/ethapi/api.go#L500-L504 --------- Co-authored-by: lightclient <lightclient@protonmail.com>
Add tests for GetBlockHeaders that verify client does not disconnect when unlikely block numbers are requested, e.g. max uint64. --------- Co-authored-by: lightclient <lightclient@protonmail.com>
The functions `rpcRequest` and `batchRpcRequest` call `baseRpcRequest`.
And `resp.Body` will be closed in the function `baseRpcRequest` later by
`t.Cleanup`:
```go
func baseRpcRequest(t *testing.T, url, bodyStr string, extraHeaders ...string) *http.Response {
// ......
t.Cleanup(func() { resp.Body.Close() })
return resp
}
```
This PR improves gas estimation for data-heavy transactions which hit the floor data gas cost.
This PR fixes the out-of-range block number logic of `getBlockLvPointer` which sometimes caused searches to fail if the head was updated in the wrong moment. This logic ensures that querying the pointer of a future block returns the pointer after the last fully indexed block (instead of failing) and therefore an async range update will not cause the search to fail. Earier this behaviour only worked when `headIndexed` was true and `headDelimiter` pointed to the end of the indexed range. Now it also works for an unfinished index. This logic is also moved from `FilterMaps.getBlockLvPointer` to `FilterMapsMatcherBackend.GetBlockLvPointer` because it is only required by the search anyways. `FilterMaps.getBlockLvPointer` now only returns a pointer for existing blocks, consistently with how it is used in the indexer/renderer. Note that this unhandled case has been present in the code for a long time but went unnoticed because either one of two previously fixed bugs did prevent it from being triggered; the incorrectly positive `tempRange.headIndexed` (fixed in ethereum/go-ethereum#31680), though caused other problems, prevented this one from being triggered as with a positive `headIndexed` no database read was triggered in `getBlockLvPointer`. Also, the unnecessary `indexLock` in `synced()` (fixed in ethereum/go-ethereum#31708) usually did prevent the search seeing the temp range and therefore avoided noticeable issues.
This PR fixes an initialization bug that in some cases caused the map renderer to leave the last, partially rendered map as is and resume rendering from the next map. At initialization we check whether the existing rendered maps are consistent with the current chain view and revert them if necessary. Until now this happened through an ugly hacky solution, a "limited" chain view that was supposed to trigger a rollback of some maps in the renderer logic if necessary. This whole setup worked under assumptions that just weren't true any more. As a result it always tried to revert the last map but also it did not shorten the indexed range, only set `headIndexed` to false which indicated to the renderer logic that the last map is fully populated (which it wasn't). Now an explicit rollback of any unusable (reorged) maps happens at startup, which also means that no hacky chain view is necessary, as soon as the new `FilterMaps` is returned, the indexed range and view are consistent with each other. In the first commit an extra check is also added to `writeFinishedMaps` so that if there is ever again a bug that would result in a gapped index then it will not break the db with writing the incomplete data. Instead it will return an indexing error which causes the indexer to revert to unindexed mode and print an error log instantly. Hopefully this will not ever happen in the future, but in order to test this safeguard check I manually triggered the bug with only the first commit enabled, which caused an indexing error as expected. With the second commit added (the actual fix) the same operation succeeded without any issues. Note that the database version is also bumped in this PR in order to enforce a full reindexing as any existing database might be potentially broken. Fixes ethereum/go-ethereum#31729
`DefaultBlobSchedule` is actually used downstream to calculate blob fees (e.g., [src](https://github.com/ethereum-optimism/optimism/blob/601a380e47853c2922ea1f8944cda05f0eac16f4/op-service/eth/blob.go#L301)), this PR makes it explicit that these params are for `Ethereum prod` instead of `test chains`.
Fixes ethereum/go-ethereum#31732. This logic was removed in the recent refactoring in the txindexer to handle history cutoff (#31393). It was first introduced in this PR: ethereum/go-ethereum#28908. I have tested it and it works as an alternative to #31745. This PR packs 3 changes to the flow of fetching txs from the API: - It caches the indexer tail after each run is over to avoid hitting the db all the time as was done originally in #28908. - Changes `backend.GetTransaction`. It doesn't return an error anymore when tx indexer is in progress. It shifts the responsibility to the caller to check the progress. The reason is that in most cases we anyway check the txpool for the tx. If it was indeed a pending tx we can avoid the indexer progress check. --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com>
…761) updates the log entries in `core/filtermaps/indexer.go` to remove double quotes around keys like "first block" and "last block", changing them to `firstblock` and `lastblock`. This brings them in line with the general logging style used across the codebase, where log keys are unquoted single words. For example, the log: ` INFO [...] "first block"=..., "last block"=...` Is now rendered as: ` INFO [...] firstblock=..., lastblock=...` This change improves readability and maintains consistency with logs such as: ` INFO [...] number=2 sealhash=... uncles=0 txs=0 ...` No functional behavior is changed — this is purely a formatting cleanup for better developer experience.
Issue statement: when user requests eth_simulateV1 to return full transaction objects, these objects always had an empty `from` field. The reason is we lose the sender when translation the message into a types.Transaction which is then later on serialized. I did think of an alternative but opted to keep with this approach as it keeps complexity at the edge. The alternative would be to pass down a signer object to RPCMarshal* methods and define a custom signer which keeps the senders in its state and doesn't attempt the signature recovery.
This change adds a limit for RPC method names to prevent potential abuse where large method names could lead to large response sizes. The limit is enforced in: - handleCall for regular RPC method calls - handleSubscribe for subscription method calls Added tests in websocket_test.go to verify the length limit functionality for both regular method calls and subscriptions. --------- Co-authored-by: Felix Lange <fjl@twurst.com>
Fixes methods debug_standardTraceBlockToFile and debug_standardTraceBadBlockToFile which were outputting empty files. --------- Co-authored-by: maskpp <maskpp266@gmail.com> Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
``` go get github.com/cockroachdb/pebble@v1.1.5 go mod tidy ``` Co-authored-by: lightclient <lightclient@protonmail.com>
This fixes an issue where blocks containing CL requests triggered an error in the engine API. The encoding of requests used base64 instead of hex.
Add metics detailing reasons we reject inbound connections for, and reasons these connections fail during the handshake.
The function `BacktraceAt` has been removed in #28187 . But the API end-point `debug_backtraceAt` is not removed from the file `internal/web3ext/web3ext.go`.
upstream: merge geth-v1.16.1
to fix: #3303 the node will periodically failed to call stakehub before the node is synced
During mining phase, if a new block was just imported, TransferPrefetcher could transfer a TriePrefetcher of previous block to the new block, which would cause later error on OpenTrieStorage of the TriePrefetch routine.
solc-0.6.4 can not be downloaded from truffle solc-bin
It is an issue reported in rs/cors#170 and fixed by rs/cors#171, which was included in release v1.11.0(Apr 2024): https://github.com/rs/cors/releases/tag/v1.11.0 But v1.8.2 was released in Dec 2021, which may have a lot difference. And this vulnerability is limitted only to RPC nodes with maliciously configuration in config.tion: [Node] HTTPCors = ["<malicious cors>",...] So instead of upgrade to v1.11.0+, simply add it to nancy ignore for now
Draft release v1.6.0-alpha
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.3)
Can you help keep this open source service alive? 💖 Please sponsor : )