Merge #7113: refactor: new chainlock::Chainlocks interface

0f1f8f1 fix: reset chainlock; make objects `chainlocks` and `clhandler` consistent (Konstantin Akimov)
a418c9f fmt: clang-format for all changes in this PR (Konstantin Akimov)
5deb316 refactor: rename llmq::CChainLocksHandler to chainlock::ChainlockHandler (Konstantin Akimov)
b57b68c fix: misusing chainlocks to determine expiring of cj's tx (Konstantin Akimov)
a894ff7 refactor: drop clhandler and chainloks from dsnotificationinterface completely (Konstantin Akimov)
040a458 refactor: don't call m_chainlocks directly in dsnotificationinterface (Konstantin Akimov)
ac22100 fix: drop dependency of CChainLocksHandler on qman; so CChainLocksHandler could be initialized before llmq_ctx (Konstantin Akimov)
a0b6483 refactor: move out VerifyChainlock from CChainLocksHandler as a function (Konstantin Akimov)
5d97d57 refactor: move clhandler from llmq_ctx to NodeContext (Konstantin Akimov)
fdfc092 fix: it should be used ChainManager inside chainlock's handler, not chainstate (Konstantin Akimov)
0d90bc1 refactor: break dependency of clhandler on chainlock/signing completely (Konstantin Akimov)
b79e2a8 refactor: chainlocks's signer to have own scheduler for cleanup and signing (Konstantin Akimov)
cf2e9b5 refactor: drop unused validation.h from spork (Konstantin Akimov)
34c05d3 refactor: move all spork usages for chainlock inside single module chainlock/chainlock (Konstantin Akimov)
3bb3433 refactor: split out lightweight chainlock::Chainlocks and widely used it interface (Konstantin Akimov)
57f661c refactor: code-move only CChainLockHandler from chainlock.h to handler.h (Konstantin Akimov)

Pull request description:

  ## Issue being fixed or feature implemented

  Currently, there are several blockers that prevent pulling consensus code out from various handlers, schedulers, mempool, etc. One of them is `CChainLocksHandler` which does too many things.

  - `CChainLocksHandler` depends on the mempool and — because it is part of `LLMQContext` — it makes `LLMQContext` depend on the mempool (which is an unexpected dependency)
  - Chainlocks depends on Instant Send, and Instant Send depends on Chainlocks (this is logically expected), but it creates a circular dependency between `CChainLocksHandler` and the initialization of these objects. See:
    `CChainLocksHandler::Start(const llmq::CInstantSendManager& isman)`
    and
    `CInstantSendManager::CInstantSendManager(CChainLocksHandler& _clhandler, ...)`
  - `CChainLocksHandler` depends on masternode-only code (`chainlock::ChainLockSigner`). It is hidden behind the interface `chainlock::ChainLockSignerParent`, but this increases implementation complexity
  - `CChainLocksHandler` is used for blocks validation (`clhandler.HasConflictingChainLock`), but at the same time `CChainLocksHandler` can switch the chain tip itself via its background worker (see calls to `EnforceBestChainLock`). This creates another dependency loop that prevents independent initialization of `CChainLocksHandler` and the Chain State.

  ## What was done?

  This PR introduces a new lightweight interface `chainlock::Chainlocks`

  - `chainlock::Chainlocks` knows about the chain state and the best known chainlock, but it does not enforce chainlocks, has no dependency on mempool or transactions, and keeps no permanent on-disk state (no dependency on evodb).
    It does depend on `SporkManager` (to know whether chainlocks are enabled/disabled) and provides a clean interface over it.

  - `llmq::CChainLocksHandler` (`chainlock::ChainlocksHandler` in future commits) is the component that actually changes Chain State by enforcing the best tip, tracks transaction's times and runs a background thread.
    It is responsible for processing newly signed chainlocks and related network messages.
    However, it no longer handles any masternode-related logic or signing — that code was fully moved to `llmq::ChainLockSigner`.

  Other less important but noticeable changes:

  - `clhandler` now uses its own subscriptions to validation notifications (no longer goes through `dsnotificationinterface`)
  - `llmq::CChainLocksHandler` was renamed and moved to a different namespace (`chainlock::ChainlocksHandler`) since it no longer belongs to `llmq::`
  - `clhandler` was removed from `llmq-ctx` and its initialization can now be skipped completely [as required for kernel project]; only `ChainLockSigner` and `PeerManager` still strongly depend on it
  - `clhandler` no longer depends on `CQuorumManager`

  Noticeable, some circular dependencies over codebase has been untangled by this PR:
   - chainlock/chainlock <-> instantsend/instantsend
   - chainlock/chainlock <-> validation
  See linter's exception list for details.

  After this PR the next step to apply similar refactoring to `CInstantSendManager` which does too many things the similar to CChainLocksHandler: keeps track of instant-send locked transactions [as expected], but also knows about masternode's code [see `instantsend::InstantSendSigner`], removes conflicting transactions from mempool, invalidates blocks in Chain State (see `CInstantSendManager::ResolveBlockConflicts`). CInstantSendManager creates logical dependencies over masternode's signer; over chainstate; over mempool and finally there are the same issues with initialization of `isman` as `clhandler` had.

  ## How Has This Been Tested?
  Run unit & functional tests

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  UdjinM6:
    utACK 0f1f8f1
  kwvg:
    utACK 0f1f8f1

Tree-SHA512: bee59b0e0e0d3530e562a8381303cf4bca721550c412ddc34edc77faee17ec02469e44f5d37fb3801fb789924b257e4dc12ac39d5ceb32087eb242c57a9bc1f3

Author: PastaPastaPasta

src/Makefile.am

@@ -174,6 +174,7 @@ BITCOIN_CORE_H = \
   blockfilter.h \
   chain.h \
   chainlock/chainlock.h \
+  chainlock/handler.h \
   chainlock/clsig.h \
   chainlock/signing.h \
   chainparams.h \
@@ -499,6 +500,7 @@ libbitcoin_node_a_SOURCES = \
   chain.cpp \
   chainlock/chainlock.cpp \
   chainlock/clsig.cpp \
+  chainlock/handler.cpp \
   chainlock/signing.cpp \
   coinjoin/coinjoin.cpp \
   coinjoin/server.cpp \

src/active/context.cpp

@@ -7,7 +7,7 @@
 #include <active/dkgsessionhandler.h>
 #include <active/masternode.h>
 #include <active/quorums.h>
-#include <chainlock/chainlock.h>
+#include <chainlock/handler.h>
 #include <chainlock/signing.h>
 #include <governance/governance.h>
 #include <governance/signing.h>
@@ -19,20 +19,19 @@
 #include <llmq/ehf_signals.h>
 #include <llmq/quorumsman.h>
 #include <llmq/signing_shares.h>
-
 #include <util/check.h>
 #include <validation.h>
+#include <validationinterface.h>
 
 ActiveContext::ActiveContext(CBLSWorker& bls_worker, ChainstateManager& chainman, CConnman& connman,
-                             CDeterministicMNManager& dmnman, CGovernanceManager& govman,
-                             CMasternodeMetaMan& mn_metaman, CMNHFManager& mnhfman, CSporkManager& sporkman,
-                             CTxMemPool& mempool, llmq::CChainLocksHandler& clhandler, llmq::CInstantSendManager& isman,
-                             llmq::CQuorumBlockProcessor& qblockman, llmq::CQuorumManager& qman,
-                             llmq::CQuorumSnapshotManager& qsnapman, llmq::CSigningManager& sigman,
-                             PeerManager& peerman, const CMasternodeSync& mn_sync, const CBLSSecretKey& operator_sk,
-                             const llmq::QvvecSyncModeMap& sync_map, const util::DbWrapperParams& db_params,
-                             bool quorums_recovery, bool quorums_watch) :
-    m_clhandler{clhandler},
+                             CDeterministicMNManager& dmnman, CGovernanceManager& govman, CMasternodeMetaMan& mn_metaman,
+                             CMNHFManager& mnhfman, CSporkManager& sporkman, const chainlock::Chainlocks& chainlocks,
+                             CTxMemPool& mempool, chainlock::ChainlockHandler& clhandler,
+                             llmq::CInstantSendManager& isman, llmq::CQuorumBlockProcessor& qblockman,
+                             llmq::CQuorumManager& qman, llmq::CQuorumSnapshotManager& qsnapman,
+                             llmq::CSigningManager& sigman, PeerManager& peerman, const CMasternodeSync& mn_sync,
+                             const CBLSSecretKey& operator_sk, const llmq::QvvecSyncModeMap& sync_map,
+                             const util::DbWrapperParams& db_params, bool quorums_recovery, bool quorums_watch) :
     m_isman{isman},
     m_qman{qman},
     nodeman{std::make_unique<CActiveMasternodeManager>(connman, dmnman, operator_sk)},
@@ -44,9 +43,9 @@ ActiveContext::ActiveContext(CBLSWorker& bls_worker, ChainstateManager& chainman
     ehf_sighandler{std::make_unique<llmq::CEHFSignalsHandler>(chainman, mnhfman, sigman, *shareman, qman)},
     qman_handler{std::make_unique<llmq::QuorumParticipant>(bls_worker, connman, dmnman, qman, qsnapman, *nodeman, chainman,
                                                            mn_sync, sporkman, sync_map, quorums_recovery, quorums_watch)},
-    cl_signer{std::make_unique<chainlock::ChainLockSigner>(chainman.ActiveChainstate(), clhandler, sigman, *shareman,
-                                                           sporkman, mn_sync)},
-    is_signer{std::make_unique<instantsend::InstantSendSigner>(chainman.ActiveChainstate(), clhandler, isman, sigman,
+    cl_signer{std::make_unique<chainlock::ChainLockSigner>(chainman.ActiveChainstate(), chainlocks, clhandler, isman,
+                                                           qman, sigman, *shareman, mn_sync)},
+    is_signer{std::make_unique<instantsend::InstantSendSigner>(chainman.ActiveChainstate(), chainlocks, isman, sigman,
                                                                *shareman, qman, sporkman, mempool, mn_sync)}
 {
     qdkgsman->InitializeHandlers([&](const Consensus::LLMQParams& llmq_params,
@@ -55,7 +54,6 @@ ActiveContext::ActiveContext(CBLSWorker& bls_worker, ChainstateManager& chainman
                                                                qblockman, qsnapman, *nodeman, chainman, sporkman,
                                                                llmq_params, quorums_watch, quorum_idx);
     });
-    m_clhandler.ConnectSigner(cl_signer.get());
     m_isman.ConnectSigner(is_signer.get());
     m_qman.ConnectManagers(qman_handler.get(), qdkgsman.get());
 }
@@ -64,7 +62,6 @@ ActiveContext::~ActiveContext()
 {
     m_qman.DisconnectManagers();
     m_isman.DisconnectSigner();
-    m_clhandler.DisconnectSigner();
 }
 
 void ActiveContext::Interrupt()
@@ -77,16 +74,22 @@ void ActiveContext::Start(CConnman& connman, PeerManager& peerman)
     qman_handler->Start();
     qdkgsman->StartThreads(connman, peerman);
     shareman->Start();
+    cl_signer->Start();
     cl_signer->RegisterRecoveryInterface();
     is_signer->RegisterRecoveryInterface();
     shareman->RegisterRecoveryInterface();
+
+    RegisterValidationInterface(cl_signer.get());
 }
 
 void ActiveContext::Stop()
 {
+    UnregisterValidationInterface(cl_signer.get());
+
     shareman->UnregisterRecoveryInterface();
     is_signer->UnregisterRecoveryInterface();
     cl_signer->UnregisterRecoveryInterface();
+    cl_signer->Stop();
     shareman->Stop();
     qdkgsman->StopThreads();
     qman_handler->Stop();

src/active/context.h

@@ -29,13 +29,14 @@ class CTxMemPool;
 class GovernanceSigner;
 class PeerManager;
 namespace chainlock {
+class Chainlocks;
+class ChainlockHandler;
 class ChainLockSigner;
 } // namespace chainlock
 namespace instantsend {
 class InstantSendSigner;
 } // namespace instantsend
 namespace llmq {
-class CChainLocksHandler;
 class CDKGDebugManager;
 class CDKGSessionManager;
 class CEHFSignalsHandler;
@@ -53,7 +54,6 @@ struct DbWrapperParams;
 
 struct ActiveContext final : public CValidationInterface {
 private:
-    llmq::CChainLocksHandler& m_clhandler;
     llmq::CInstantSendManager& m_isman;
     llmq::CQuorumManager& m_qman;
 
@@ -63,13 +63,13 @@ struct ActiveContext final : public CValidationInterface {
     ActiveContext& operator=(const ActiveContext&) = delete;
     explicit ActiveContext(CBLSWorker& bls_worker, ChainstateManager& chainman, CConnman& connman,
                            CDeterministicMNManager& dmnman, CGovernanceManager& govman, CMasternodeMetaMan& mn_metaman,
-                           CMNHFManager& mnhfman, CSporkManager& sporkman, CTxMemPool& mempool,
-                           llmq::CChainLocksHandler& clhandler, llmq::CInstantSendManager& isman,
-                           llmq::CQuorumBlockProcessor& qblockman, llmq::CQuorumManager& qman,
-                           llmq::CQuorumSnapshotManager& qsnapman, llmq::CSigningManager& sigman, PeerManager& peerman,
-                           const CMasternodeSync& mn_sync, const CBLSSecretKey& operator_sk,
-                           const llmq::QvvecSyncModeMap& sync_map, const util::DbWrapperParams& db_params,
-                           bool quorums_recovery, bool quorums_watch);
+                           CMNHFManager& mnhfman, CSporkManager& sporkman, const chainlock::Chainlocks& chainlocks,
+                           CTxMemPool& mempool, chainlock::ChainlockHandler& clhandler,
+                           llmq::CInstantSendManager& isman, llmq::CQuorumBlockProcessor& qblockman,
+                           llmq::CQuorumManager& qman, llmq::CQuorumSnapshotManager& qsnapman,
+                           llmq::CSigningManager& sigman, PeerManager& peerman, const CMasternodeSync& mn_sync,
+                           const CBLSSecretKey& operator_sk, const llmq::QvvecSyncModeMap& sync_map,
+                           const util::DbWrapperParams& db_params, bool quorums_recovery, bool quorums_watch);
     ~ActiveContext();
 
     void Interrupt();

src/bench/rpc_blockchain.cpp

@@ -45,7 +45,7 @@ static void BlockToJsonVerbose(benchmark::Bench& bench)
     TestBlockAndIndex data;
     const LLMQContext& llmq_ctx = *data.testing_setup->m_node.llmq_ctx;
     bench.run([&] {
-        auto univalue = blockToJSON(data.testing_setup->m_node.chainman->m_blockman, data.block, &data.blockindex, &data.blockindex, *llmq_ctx.clhandler, *llmq_ctx.isman, TxVerbosity::SHOW_DETAILS_AND_PREVOUT);
+        auto univalue = blockToJSON(data.testing_setup->m_node.chainman->m_blockman, data.block, &data.blockindex, &data.blockindex, *data.testing_setup->m_node.chainlocks, *llmq_ctx.isman, TxVerbosity::SHOW_DETAILS_AND_PREVOUT);
         ankerl::nanobench::doNotOptimizeAway(univalue);
     });
 }
@@ -56,7 +56,7 @@ static void BlockToJsonVerboseWrite(benchmark::Bench& bench)
 {
     TestBlockAndIndex data;
     const LLMQContext& llmq_ctx = *data.testing_setup->m_node.llmq_ctx;
-    auto univalue = blockToJSON(data.testing_setup->m_node.chainman->m_blockman, data.block, &data.blockindex, &data.blockindex, *llmq_ctx.clhandler, *llmq_ctx.isman, TxVerbosity::SHOW_DETAILS_AND_PREVOUT);
+    auto univalue = blockToJSON(data.testing_setup->m_node.chainman->m_blockman, data.block, &data.blockindex, &data.blockindex, *data.testing_setup->m_node.chainlocks, *llmq_ctx.isman, TxVerbosity::SHOW_DETAILS_AND_PREVOUT);
     bench.run([&] {
         auto str = univalue.write();
         ankerl::nanobench::doNotOptimizeAway(str);