Skip to content

rc/supernova #353

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
sstanculeanu wants to merge 20 commits into
base: master
Choose a base branch
from
Draft

rc/supernova #353

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
bf86b96
writes in parallel
miiu96 Jul 1, 2025
806739d
fixes after review
miiu96 Jul 29, 2025
97a4a3e
Merge pull request #349 from multiversx/writes-in-parallel
miiu96 Sep 1, 2025
bd6aa66
Merge branch 'master' of https://github.com/multiversx/mx-chain-es-in…
sstanculeanu Oct 21, 2025
7c00b19
Merge pull request #354 from multiversx/merge-master-into-rc/supernov…
sstanculeanu Oct 22, 2025
0ec2067
execution results index
miiu96 Nov 6, 2025
ec0eecd
prepare execution results data
miiu96 Nov 11, 2025
118a2cd
fix unit tests
miiu96 Nov 11, 2025
f5e03ab
fix integration tests
miiu96 Nov 11, 2025
f703a4b
index and remove execution results
miiu96 Nov 11, 2025
19221ee
refactoring tx processor
miiu96 Nov 12, 2025
3031b52
index data from execution results
miiu96 Nov 13, 2025
548bed2
refactoring
miiu96 Nov 18, 2025
721a9b5
remove
miiu96 Nov 18, 2025
361ad19
fix integration tests
miiu96 Nov 18, 2025
0d25c5c
execution results integration test
miiu96 Nov 18, 2025
1cdeb23
unit tests and fixes
miiu96 Nov 18, 2025
e639212
fixes
miiu96 Nov 18, 2025
745300b
fixes after review
miiu96 Nov 20, 2025
7a403d4
Merge pull request #355 from multiversx/execution-results-index
miiu96 Nov 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cmd/elasticindexer/config/config.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
available-indices = [
"rating", "transactions", "blocks", "validators", "miniblocks", "rounds", "accounts", "accountshistory",
"receipts", "scresults", "accountsesdt", "accountsesdthistory", "epochinfo", "scdeploys", "tokens", "tags",
"logs", "delegators", "operations", "esdts", "values", "events"
"logs", "delegators", "operations", "esdts", "values", "events", "executionresults"
]
[config.address-converter]
length = 32
Expand Down
1 change: 1 addition & 0 deletions cmd/elasticindexer/config/prefs.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,4 @@
username = ""
password = ""
bulk-request-max-size-in-bytes = 4194304 # 4MB
num-writes-in-parallel = 1
1 change: 1 addition & 0 deletions config/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ type ClusterConfig struct {
UserName string `toml:"username"`
Password string `toml:"password"`
BulkRequestMaxSizeInBytes int `toml:"bulk-request-max-size-in-bytes"`
NumWritesInParallel int `toml:"num-writes-in-parallel"`
} `toml:"elastic-cluster"`
} `toml:"config"`
}
Expand Down
121 changes: 80 additions & 41 deletions data/block.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package data

import (
coreData "github.com/multiversx/mx-chain-core-go/data"
"github.com/multiversx/mx-chain-core-go/data/api"
)

Expand All @@ -9,47 +10,85 @@ import (
// to be saved for a block. It has all the default fields
// plus some extra information for ease of search and filter
type Block struct {
UUID string `json:"uuid"`
Nonce uint64 `json:"nonce"`
Round uint64 `json:"round"`
Epoch uint32 `json:"epoch"`
Hash string `json:"-"`
MiniBlocksHashes []string `json:"miniBlocksHashes"`
MiniBlocksDetails []*MiniBlocksDetails `json:"miniBlocksDetails,omitempty"`
NotarizedBlocksHashes []string `json:"notarizedBlocksHashes"`
Proposer uint64 `json:"proposer"`
Validators []uint64 `json:"validators,omitempty"`
PubKeyBitmap string `json:"pubKeyBitmap"`
Size int64 `json:"size"`
SizeTxs int64 `json:"sizeTxs"`
Timestamp uint64 `json:"timestamp"`
TimestampMs uint64 `json:"timestampMs,omitempty"`
StateRootHash string `json:"stateRootHash"`
PrevHash string `json:"prevHash"`
ShardID uint32 `json:"shardId"`
TxCount uint32 `json:"txCount"`
NotarizedTxsCount uint32 `json:"notarizedTxsCount"`
AccumulatedFees string `json:"accumulatedFees"`
DeveloperFees string `json:"developerFees"`
EpochStartBlock bool `json:"epochStartBlock"`
SearchOrder uint64 `json:"searchOrder"`
EpochStartInfo *EpochStartInfo `json:"epochStartInfo,omitempty"`
GasProvided uint64 `json:"gasProvided"`
GasRefunded uint64 `json:"gasRefunded"`
GasPenalized uint64 `json:"gasPenalized"`
MaxGasLimit uint64 `json:"maxGasLimit"`
ScheduledData *ScheduledData `json:"scheduledData,omitempty"`
EpochStartShardsData []*EpochStartShardData `json:"epochStartShardsData,omitempty"`
Proof *api.HeaderProof `json:"proof,omitempty"`
RandSeed string `json:"randSeed,omitempty"`
PrevRandSeed string `json:"prevRandSeed,omitempty"`
Signature string `json:"signature,omitempty"`
LeaderSignature string `json:"leaderSignature,omitempty"`
ChainID string `json:"chainID,omitempty"`
SoftwareVersion string `json:"softwareVersion,omitempty"`
ReceiptsHash string `json:"receiptsHash,omitempty"`
Reserved []byte `json:"reserved,omitempty"`
ProposerBlsKey string `json:"proposerBlsKey,omitempty"`
UUID string `json:"uuid"`
Nonce uint64 `json:"nonce"`
Round uint64 `json:"round"`
Epoch uint32 `json:"epoch"`
Hash string `json:"-"`
MiniBlocksHashes []string `json:"miniBlocksHashes"`
MiniBlocksDetails []*MiniBlocksDetails `json:"miniBlocksDetails,omitempty"`
NotarizedBlocksHashes []string `json:"notarizedBlocksHashes"`
Proposer uint64 `json:"proposer"`
Validators []uint64 `json:"validators,omitempty"`
PubKeyBitmap string `json:"pubKeyBitmap"`
Size int64 `json:"size"`
SizeTxs int64 `json:"sizeTxs"`
Timestamp uint64 `json:"timestamp"`
TimestampMs uint64 `json:"timestampMs,omitempty"`
StateRootHash string `json:"stateRootHash"`
PrevHash string `json:"prevHash"`
ShardID uint32 `json:"shardId"`
TxCount uint32 `json:"txCount"`
NotarizedTxsCount uint32 `json:"notarizedTxsCount"`
AccumulatedFees string `json:"accumulatedFees"`
DeveloperFees string `json:"developerFees"`
EpochStartBlock bool `json:"epochStartBlock"`
SearchOrder uint64 `json:"searchOrder"`
EpochStartInfo *EpochStartInfo `json:"epochStartInfo,omitempty"`
GasProvided uint64 `json:"gasProvided"`
GasRefunded uint64 `json:"gasRefunded"`
GasPenalized uint64 `json:"gasPenalized"`
MaxGasLimit uint64 `json:"maxGasLimit"`
ScheduledData *ScheduledData `json:"scheduledData,omitempty"`
EpochStartShardsData []*EpochStartShardData `json:"epochStartShardsData,omitempty"`
Proof *api.HeaderProof `json:"proof,omitempty"`
RandSeed string `json:"randSeed,omitempty"`
PrevRandSeed string `json:"prevRandSeed,omitempty"`
Signature string `json:"signature,omitempty"`
LeaderSignature string `json:"leaderSignature,omitempty"`
ChainID string `json:"chainID,omitempty"`
SoftwareVersion string `json:"softwareVersion,omitempty"`
ReceiptsHash string `json:"receiptsHash,omitempty"`
Reserved []byte `json:"reserved,omitempty"`
ProposerBlsKey string `json:"proposerBlsKey,omitempty"`
ExecutionResultBlockHashes []string `json:"executionResultBlockHashes,omitempty"`
}

// ExecutionResult is a structure containing all the fields that need
//
// to be saved for an execution results.
type ExecutionResult struct {
UUID string `json:"uuid"`
Hash string `json:"-"`
RootHash string `json:"rootHash"`
NotarizedInBlockHash string `json:"notarizedInBlockHash"`
AccumulatedFees string `json:"accumulatedFees"`
DeveloperFees string `json:"developerFees"`
TxCount uint64 `json:"txCount"`
GasUsed uint64 `json:"gasUsed"`
Nonce uint64 `json:"nonce"`
Round uint64 `json:"round"`
Epoch uint32 `json:"epoch"`
MiniBlocksHashes []string `json:"miniBlocksHashes"`
MiniBlocksDetails []*MiniBlocksDetails `json:"miniBlocksDetails,omitempty"`
}

// HeaderData is the DTO that holds fields from header or execution result
type HeaderData struct {
Timestamp uint64
TimestampMs uint64
Round uint64
ShardID uint32
NumberOfShards uint32
Epoch uint32
HeaderHash []byte
MiniBlockHeaders []coreData.MiniBlockHeaderHandler
}

// PreparedBlockResults is the DTO that holds all the results after processing a block
type PreparedBlockResults struct {
Block *Block
ExecutionResults []*ExecutionResult
}

// MiniBlocksDetails is a structure that hold information about mini-blocks execution details
Expand Down
1 change: 1 addition & 0 deletions factory/wsIndexerFactory.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ func createDataIndexer(
StatusMetrics: statusMetrics,
Version: version,
EnableEpochsConfig: enableEpochsCfg,
NumWritesInParallel: clusterCfg.Config.ElasticCluster.NumWritesInParallel,
})
}

Expand Down
5 changes: 3 additions & 2 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ go 1.23

require (
github.com/elastic/go-elasticsearch/v7 v7.12.0
github.com/gin-contrib/cors v1.4.0
github.com/gin-contrib/cors v1.6.0
github.com/gin-gonic/gin v1.10.0
github.com/google/uuid v1.6.0
github.com/multiversx/mx-chain-communication-go v1.3.0
github.com/multiversx/mx-chain-core-go v1.4.0
github.com/multiversx/mx-chain-core-go v1.4.2-0.20251113093212-347994c38d2d
github.com/multiversx/mx-chain-logger-go v1.1.0
github.com/multiversx/mx-chain-vm-common-go v1.6.0
github.com/prometheus/client_model v0.6.1
Expand Down Expand Up @@ -40,6 +40,7 @@ require (
github.com/gorilla/websocket v1.5.3 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.2.9 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
Expand Down
Loading
Loading