Skip to content

Commit e3b88cc

Browse files
PastaPastaPastaPastaPastaPasta
committed
Merge #6678: perf: use vector instead CDeterministicMNList for computing rotating quorums
2389d97 refactor: remove un-needed std::move from GetQuorumQuarterMembersBySnapshot (Konstantin Akimov) 7ab8647 perf: move std::vector of masternodes instead copying during rotation quorum calculation (Konstantin Akimov) 026847a refactor: add default argument maxSize for CalcualeQuorum (Konstantin Akimov) 291ea2e refactor: remove useless variable copy (even with move) (Konstantin Akimov) fd901e0 perf: futher using std::vector instead CDeterministicMNList (Konstantin Akimov) 80faf1b perf: use vector instead DeterministicMNList for calculation of rotating quorums (Konstantin Akimov) 77273d5 perf: use std::move during quorum calculation (Konstantin Akimov) Pull request description: ## Issue being fixed or feature implemented Helper to get quorum members uses CDeterministicMNList as a source of data. It makes temporary calculation slower, because temporary lists of masternodes should not construct heavy objects CDeterministicMNList just to keep a list of them. ## What was done? - helpers `CalculateQuorum`, `CalculateScores` are moved from class `CDeterministicMNList` to `llmq::utils` so far as they don't actually use any private data of `CDeterministicMNList` - helper `CalculateQuorum` can accept as source of data not only CDeterministicMNList but std::vector with shared ptrs. - helper `BuildNewQuorumQuarterMembers` does not use CDeterministicMNList for `MnsNotUsedAtH` - `GetMNUsageBySnapshot` does not use CDeterministicMNList anymore Also, `BuildNewQuorumQuarterMembers` uses `std::move` when possible. ## How Has This Been Tested? invalidate + reconsider 15000 blocks; check logs and `perf`. By perf `PreComputeQuorumMembers` got almost double faster; most improvements came from `GetQuorumQuarterMembersBySnapshot` as expected. It should give roughly 2% overall improvement for block validation speed and reindex. PR: <img width="747" alt="image" src="https://github.com/user-attachments/assets/d27778fa-cc09-4bd7-bccd-601fa6df75f3" /> ``` 2025-05-20T15:46:53Z [bench] - m_qblockman: 0.05ms [87.08s] 2025-05-20T15:46:53Z [bench] - Connect block: 21.17ms [249.92s (17.38ms/blk)] ``` Develop: <img width="747" alt="image" src="https://github.com/user-attachments/assets/aff54059-703d-4c76-967a-15279b1b420c" /> ``` 2025-05-20T16:11:33Z [bench] - m_qblockman: 0.05ms [93.42s] 2025-05-20T16:11:33Z [bench] - Connect block: 19.06ms [256.95s (17.13ms/blk)] ``` ## Breaking Changes N/A ## Checklist: - [x] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have added or updated relevant unit/integration/functional/e2e tests - [ ] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone ACKs for top commit: UdjinM6: utACK 2389d97 PastaPastaPasta: utACK 2389d97 Tree-SHA512: 86db57e74eb815cc030421610d20e2dd90dfd18fcf429f97ba96681b69539cce54b0f4c9146e29beb541d1e0193919a1242bd08cba8df7017cfe4b42b8be7a63
2 parents 9ef4783 + 2389d97 commit e3b88cc

File tree

3 files changed

+116
-93
lines changed

3 files changed

+116
-93
lines changed

‎src/evo/deterministicmns.cpp‎

Lines changed: 0 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -255,58 +255,6 @@ std::vector<CDeterministicMNCPtr> CDeterministicMNList::GetProjectedMNPayees(gsl
255255
return result;
256256
}
257257

258-
std::vector<CDeterministicMNCPtr> CDeterministicMNList::CalculateQuorum(size_t maxSize, const uint256& modifier, const bool onlyEvoNodes) const
259-
{
260-
auto scores = CalculateScores(modifier, onlyEvoNodes);
261-
262-
// sort is descending order
263-
std::sort(scores.rbegin(), scores.rend(), [](const std::pair<arith_uint256, CDeterministicMNCPtr>& a, const std::pair<arith_uint256, CDeterministicMNCPtr>& b) {
264-
if (a.first == b.first) {
265-
// this should actually never happen, but we should stay compatible with how the non-deterministic MNs did the sorting
266-
return a.second->collateralOutpoint < b.second->collateralOutpoint;
267-
}
268-
return a.first < b.first;
269-
});
270-
271-
// take top maxSize entries and return it
272-
std::vector<CDeterministicMNCPtr> result;
273-
result.resize(std::min(maxSize, scores.size()));
274-
for (size_t i = 0; i < result.size(); i++) {
275-
result[i] = std::move(scores[i].second);
276-
}
277-
return result;
278-
}
279-
280-
std::vector<std::pair<arith_uint256, CDeterministicMNCPtr>> CDeterministicMNList::CalculateScores(const uint256& modifier, const bool onlyEvoNodes) const
281-
{
282-
std::vector<std::pair<arith_uint256, CDeterministicMNCPtr>> scores;
283-
scores.reserve(GetAllMNsCount());
284-
ForEachMNShared(true, [&](const CDeterministicMNCPtr& dmn) {
285-
if (dmn->pdmnState->confirmedHash.IsNull()) {
286-
// we only take confirmed MNs into account to avoid hash grinding on the ProRegTxHash to sneak MNs into a
287-
// future quorums
288-
return;
289-
}
290-
if (onlyEvoNodes) {
291-
if (dmn->nType != MnType::Evo)
292-
return;
293-
}
294-
// calculate sha256(sha256(proTxHash, confirmedHash), modifier) per MN
295-
// Please note that this is not a double-sha256 but a single-sha256
296-
// The first part is already precalculated (confirmedHashWithProRegTxHash)
297-
// TODO When https://github.com/bitcoin/bitcoin/pull/13191 gets backported, implement something that is similar but for single-sha256
298-
uint256 h;
299-
CSHA256 sha256;
300-
sha256.Write(dmn->pdmnState->confirmedHashWithProRegTxHash.begin(), dmn->pdmnState->confirmedHashWithProRegTxHash.size());
301-
sha256.Write(modifier.begin(), modifier.size());
302-
sha256.Finalize(h.begin());
303-
304-
scores.emplace_back(UintToArith256(h), dmn);
305-
});
306-
307-
return scores;
308-
}
309-
310258
int CDeterministicMNList::CalcMaxPoSePenalty() const
311259
{
312260
// Maximum PoSe penalty is dynamic and equals the number of registered MNs

‎src/evo/deterministicmns.h‎

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -331,12 +331,6 @@ class CDeterministicMNList
331331
*/
332332
[[nodiscard]] std::vector<CDeterministicMNCPtr> GetProjectedMNPayees(gsl::not_null<const CBlockIndex* const> pindexPrev, int nCount = std::numeric_limits<int>::max()) const;
333333

334-
/**
335-
* Calculate a quorum based on the modifier. The resulting list is deterministically sorted by score
336-
*/
337-
[[nodiscard]] std::vector<CDeterministicMNCPtr> CalculateQuorum(size_t maxSize, const uint256& modifier, const bool onlyEvoNodes = false) const;
338-
[[nodiscard]] std::vector<std::pair<arith_uint256, CDeterministicMNCPtr>> CalculateScores(const uint256& modifier, const bool onlyEvoNodes) const;
339-
340334
/**
341335
* Calculates the maximum penalty which is allowed at the height of this MN list. It is dynamic and might change
342336
* for every block.

‎src/llmq/utils.cpp‎

Lines changed: 116 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ static PreviousQuorumQuarters GetPreviousQuorumQuarterMembers(const Consensus::L
7575
static std::vector<std::vector<CDeterministicMNCPtr>> GetQuorumQuarterMembersBySnapshot(
7676
const Consensus::LLMQParams& llmqParams, CDeterministicMNManager& dmnman,
7777
const CBlockIndex* pCycleQuorumBaseBlockIndex, const llmq::CQuorumSnapshot& snapshot, int nHeight);
78-
static std::pair<CDeterministicMNList, CDeterministicMNList> GetMNUsageBySnapshot(
78+
static std::pair<std::vector<CDeterministicMNCPtr>, std::vector<CDeterministicMNCPtr>> GetMNUsageBySnapshot(
7979
const Consensus::LLMQParams& llmqParams, CDeterministicMNManager& dmnman,
8080
const CBlockIndex* pCycleQuorumBaseBlockIndex, const llmq::CQuorumSnapshot& snapshot, int nHeight);
8181

@@ -108,6 +108,98 @@ static uint256 GetHashModifier(const Consensus::LLMQParams& llmqParams, gsl::not
108108
return ::SerializeHash(std::make_pair(llmqParams.type, pCycleQuorumBaseBlockIndex->GetBlockHash()));
109109
}
110110

111+
static arith_uint256 calculateQuorumScore(const CDeterministicMNCPtr& dmn, const uint256& modifier)
112+
{
113+
// calculate sha256(sha256(proTxHash, confirmedHash), modifier) per MN
114+
// Please note that this is not a double-sha256 but a single-sha256
115+
// The first part is already precalculated (confirmedHashWithProRegTxHash)
116+
// TODO When https://github.com/bitcoin/bitcoin/pull/13191 gets backported, implement something that is similar but for single-sha256
117+
uint256 h;
118+
CSHA256 sha256;
119+
sha256.Write(dmn->pdmnState->confirmedHashWithProRegTxHash.begin(),
120+
dmn->pdmnState->confirmedHashWithProRegTxHash.size());
121+
sha256.Write(modifier.begin(), modifier.size());
122+
sha256.Finalize(h.begin());
123+
return UintToArith256(h);
124+
}
125+
126+
static std::vector<std::pair<arith_uint256, CDeterministicMNCPtr>> CalculateScoresForQuorum(
127+
std::vector<CDeterministicMNCPtr>&& dmns, const uint256& modifier, const bool onlyEvoNodes)
128+
{
129+
std::vector<std::pair<arith_uint256, CDeterministicMNCPtr>> scores;
130+
scores.reserve(dmns.size());
131+
132+
for (auto& dmn : dmns) {
133+
if (dmn->pdmnState->IsBanned()) continue;
134+
if (dmn->pdmnState->confirmedHash.IsNull()) {
135+
// we only take confirmed MNs into account to avoid hash grinding on the ProRegTxHash to sneak MNs into a
136+
// future quorums
137+
continue;
138+
}
139+
if (onlyEvoNodes && dmn->nType != MnType::Evo) {
140+
continue;
141+
}
142+
scores.emplace_back(calculateQuorumScore(dmn, modifier), std::move(dmn));
143+
};
144+
return scores;
145+
}
146+
147+
static std::vector<std::pair<arith_uint256, CDeterministicMNCPtr>> CalculateScoresForQuorum(
148+
const CDeterministicMNList& mn_list, const uint256& modifier, const bool onlyEvoNodes)
149+
{
150+
std::vector<std::pair<arith_uint256, CDeterministicMNCPtr>> scores;
151+
scores.reserve(mn_list.GetAllMNsCount());
152+
153+
mn_list.ForEachMNShared(true, [&](const CDeterministicMNCPtr& dmn) {
154+
if (dmn->pdmnState->confirmedHash.IsNull()) {
155+
// we only take confirmed MNs into account to avoid hash grinding on the ProRegTxHash to sneak MNs into a
156+
// future quorums
157+
return;
158+
}
159+
if (onlyEvoNodes && dmn->nType != MnType::Evo) {
160+
return;
161+
}
162+
163+
scores.emplace_back(calculateQuorumScore(dmn, modifier), dmn);
164+
});
165+
return scores;
166+
}
167+
168+
169+
/**
170+
* Calculate a quorum based on the modifier. The resulting list is deterministically sorted by score
171+
*/
172+
template <typename List>
173+
static std::vector<CDeterministicMNCPtr> CalculateQuorum(List&& mn_list, const uint256& modifier, size_t maxSize = 0,
174+
const bool onlyEvoNodes = false)
175+
{
176+
auto scores = CalculateScoresForQuorum(std::forward<List>(mn_list), modifier, onlyEvoNodes);
177+
178+
// sort is descending order
179+
std::sort(scores.rbegin(), scores.rend(),
180+
[](const std::pair<arith_uint256, CDeterministicMNCPtr>& a,
181+
const std::pair<arith_uint256, CDeterministicMNCPtr>& b) {
182+
if (a.first == b.first) {
183+
// this should actually never happen, but we should stay compatible with how the non-deterministic MNs did the sorting
184+
// TODO - add assert ?
185+
return a.second->collateralOutpoint < b.second->collateralOutpoint;
186+
}
187+
return a.first < b.first;
188+
});
189+
190+
// return top maxSize entries only (if specified)
191+
if (maxSize > 0 && scores.size() > maxSize) {
192+
scores.resize(maxSize);
193+
}
194+
195+
std::vector<CDeterministicMNCPtr> result;
196+
result.reserve(scores.size());
197+
for (auto& score : scores) {
198+
result.emplace_back(std::move(score.second));
199+
}
200+
return result;
201+
}
202+
111203
std::vector<CDeterministicMNCPtr> GetAllQuorumMembers(Consensus::LLMQType llmqType, CDeterministicMNManager& dmnman,
112204
CQuorumSnapshotManager& qsnapman,
113205
gsl::not_null<const CBlockIndex*> pQuorumBaseBlockIndex,
@@ -171,12 +263,13 @@ std::vector<CDeterministicMNCPtr> GetAllQuorumMembers(Consensus::LLMQType llmqTy
171263
}
172264

173265
auto q = ComputeQuorumMembersByQuarterRotation(llmq_params, dmnman, qsnapman, pCycleQuorumBaseBlockIndex);
266+
quorumMembers = q[quorumIndex];
267+
174268
LOCK(cs_indexed_members);
175269
for (const size_t i : irange::range(q.size())) {
176-
mapIndexedQuorumMembers[llmqType].insert(std::make_pair(pCycleQuorumBaseBlockIndex->GetBlockHash(), i), q[i]);
270+
mapIndexedQuorumMembers[llmqType].emplace(std::make_pair(pCycleQuorumBaseBlockIndex->GetBlockHash(), i),
271+
std::move(q[i]));
177272
}
178-
179-
quorumMembers = q[quorumIndex];
180273
} else {
181274
quorumMembers = ComputeQuorumMembers(llmqType, dmnman, pQuorumBaseBlockIndex);
182275
}
@@ -202,7 +295,7 @@ std::vector<CDeterministicMNCPtr> ComputeQuorumMembers(Consensus::LLMQType llmqT
202295
pQuorumBaseBlockIndex;
203296
const auto modifier = GetHashModifier(llmq_params_opt.value(), pQuorumBaseBlockIndex);
204297
auto allMns = dmnman.GetListForBlock(pWorkBlockIndex);
205-
return allMns.CalculateQuorum(llmq_params_opt->size, modifier, EvoOnly);
298+
return CalculateQuorum(allMns, modifier, llmq_params_opt->size, EvoOnly);
206299
}
207300

208301
std::vector<std::vector<CDeterministicMNCPtr>> ComputeQuorumMembersByQuarterRotation(
@@ -345,7 +438,6 @@ std::vector<std::vector<CDeterministicMNCPtr>> BuildNewQuorumQuarterMembers(
345438
}
346439

347440
auto MnsUsedAtH = CDeterministicMNList();
348-
auto MnsNotUsedAtH = CDeterministicMNList();
349441
std::vector<CDeterministicMNList> MnsUsedAtHIndexed{nQuorums};
350442

351443
bool skipRemovedMNs = IsV19Active(pCycleQuorumBaseBlockIndex) || (Params().NetworkIDString() == CBaseChainParams::TESTNET);
@@ -401,20 +493,17 @@ std::vector<std::vector<CDeterministicMNCPtr>> BuildNewQuorumQuarterMembers(
401493
}
402494
}
403495

496+
std::vector<CDeterministicMNCPtr> MnsNotUsedAtH;
404497
allMns.ForEachMNShared(false, [&MnsUsedAtH, &MnsNotUsedAtH](const CDeterministicMNCPtr& dmn) {
405498
if (!MnsUsedAtH.HasMN(dmn->proTxHash)) {
406499
if (!dmn->pdmnState->IsBanned()) {
407-
try {
408-
MnsNotUsedAtH.AddMN(dmn);
409-
} catch (const std::runtime_error& e) {
410-
}
500+
MnsNotUsedAtH.push_back(dmn);
411501
}
412502
}
413503
});
414504

415-
auto sortedMnsUsedAtHM = MnsUsedAtH.CalculateQuorum(MnsUsedAtH.GetAllMNsCount(), modifier);
416-
auto sortedMnsNotUsedAtH = MnsNotUsedAtH.CalculateQuorum(MnsNotUsedAtH.GetAllMNsCount(), modifier);
417-
auto sortedCombinedMnsList = std::move(sortedMnsNotUsedAtH);
505+
auto sortedMnsUsedAtHM = CalculateQuorum(MnsUsedAtH, modifier);
506+
auto sortedCombinedMnsList = CalculateQuorum(std::move(MnsNotUsedAtH), modifier);
418507
for (auto& m : sortedMnsUsedAtHM) {
419508
sortedCombinedMnsList.push_back(std::move(m));
420509
}
@@ -493,7 +582,7 @@ void BuildQuorumSnapshot(const Consensus::LLMQParams& llmqParams, const CDetermi
493582

494583
quorumSnapshot.activeQuorumMembers.resize(allMns.GetAllMNsCount());
495584
const auto modifier = GetHashModifier(llmqParams, pCycleQuorumBaseBlockIndex);
496-
auto sortedAllMns = allMns.CalculateQuorum(allMns.GetAllMNsCount(), modifier);
585+
auto sortedAllMns = CalculateQuorum(allMns, modifier);
497586

498587
LogPrint(BCLog::LLMQ, "BuildQuorumSnapshot h[%d] numMns[%d]\n", pCycleQuorumBaseBlockIndex->nHeight, allMns.GetAllMNsCount());
499588

@@ -529,12 +618,12 @@ std::vector<std::vector<CDeterministicMNCPtr>> GetQuorumQuarterMembersBySnapshot
529618
std::vector<CDeterministicMNCPtr> sortedCombinedMns;
530619
{
531620
const auto modifier = GetHashModifier(llmqParams, pCycleQuorumBaseBlockIndex);
532-
const auto [MnsUsedAtH, MnsNotUsedAtH] = GetMNUsageBySnapshot(llmqParams, dmnman, pCycleQuorumBaseBlockIndex, snapshot, nHeight);
621+
auto [MnsUsedAtH, MnsNotUsedAtH] = GetMNUsageBySnapshot(llmqParams, dmnman, pCycleQuorumBaseBlockIndex,
622+
snapshot, nHeight);
533623
// the list begins with all the unused MNs
534-
auto sortedMnsNotUsedAtH = MnsNotUsedAtH.CalculateQuorum(MnsNotUsedAtH.GetAllMNsCount(), modifier);
535-
sortedCombinedMns = std::move(sortedMnsNotUsedAtH);
624+
sortedCombinedMns = CalculateQuorum(std::move(MnsNotUsedAtH), modifier);
536625
// Now add the already used MNs to the end of the list
537-
auto sortedMnsUsedAtH = MnsUsedAtH.CalculateQuorum(MnsUsedAtH.GetAllMNsCount(), modifier);
626+
auto sortedMnsUsedAtH = CalculateQuorum(std::move(MnsUsedAtH), modifier);
538627
std::move(sortedMnsUsedAtH.begin(), sortedMnsUsedAtH.end(), std::back_inserter(sortedCombinedMns));
539628
}
540629

@@ -611,45 +700,37 @@ std::vector<std::vector<CDeterministicMNCPtr>> GetQuorumQuarterMembersBySnapshot
611700
}
612701
}
613702

614-
std::pair<CDeterministicMNList, CDeterministicMNList> GetMNUsageBySnapshot(const Consensus::LLMQParams& llmqParams,
615-
CDeterministicMNManager& dmnman,
616-
const CBlockIndex* pCycleQuorumBaseBlockIndex,
617-
const llmq::CQuorumSnapshot& snapshot,
618-
int nHeight)
703+
static std::pair<std::vector<CDeterministicMNCPtr>, std::vector<CDeterministicMNCPtr>> GetMNUsageBySnapshot(
704+
const Consensus::LLMQParams& llmqParams, CDeterministicMNManager& dmnman,
705+
const CBlockIndex* pCycleQuorumBaseBlockIndex, const llmq::CQuorumSnapshot& snapshot, int nHeight)
619706
{
620707
if (!llmqParams.useRotation || pCycleQuorumBaseBlockIndex->nHeight % llmqParams.dkgInterval != 0) {
621708
ASSERT_IF_DEBUG(false);
622709
return {};
623710
}
624711

625-
CDeterministicMNList usedMNs;
626-
CDeterministicMNList nonUsedMNs;
712+
std::vector<CDeterministicMNCPtr> usedMNs;
713+
std::vector<CDeterministicMNCPtr> nonUsedMNs;
627714

628715
const CBlockIndex* pWorkBlockIndex = pCycleQuorumBaseBlockIndex->GetAncestor(pCycleQuorumBaseBlockIndex->nHeight - 8);
629716
const auto modifier = GetHashModifier(llmqParams, pCycleQuorumBaseBlockIndex);
630717

631718
auto allMns = dmnman.GetListForBlock(pWorkBlockIndex);
632-
auto sortedAllMns = allMns.CalculateQuorum(allMns.GetAllMNsCount(), modifier);
719+
auto sortedAllMns = CalculateQuorum(allMns, modifier);
633720

634721
size_t i{0};
635722
for (const auto& dmn : sortedAllMns) {
636723
if (snapshot.activeQuorumMembers[i]) {
637-
try {
638-
usedMNs.AddMN(dmn);
639-
} catch (const std::runtime_error& e) {
640-
}
724+
usedMNs.push_back(dmn);
641725
} else {
642726
if (!dmn->pdmnState->IsBanned()) {
643-
try {
644-
nonUsedMNs.AddMN(dmn);
645-
} catch (const std::runtime_error& e) {
646-
}
727+
nonUsedMNs.push_back(dmn);
647728
}
648729
}
649730
i++;
650731
}
651732

652-
return std::make_pair(usedMNs, nonUsedMNs);
733+
return {usedMNs, nonUsedMNs};
653734
}
654735

655736
uint256 DeterministicOutboundConnection(const uint256& proTxHash1, const uint256& proTxHash2)

0 commit comments

Comments
 (0)