Merge #13247: Add tests to SingleThreadedSchedulerClient() and document the memory model

cbeaa91 Update ValidationInterface() documentation to explicitly specify threading and memory model (Jesse Cohen)
b296b42 Update documentation for SingleThreadedSchedulerClient() to specify the memory model (Jesse Cohen)
9994d01 Add Unit Test for SingleThreadedSchedulerClient (Jesse Cohen)

Pull request description:

  As discussed in #13023 I've split this test out into a separate pr

  This test (and documentation update) makes explicit the guarantee (previously undefined, but implied by the 'SingleThreaded' in `SingleThreadedSchedulerClient()`) - that callbacks pushed to the `SingleThreadedSchedulerClient()` obey the single threaded model for memory and execution - specifically, the callbacks are executed fully and in order, and even in cases where a subsequent callback is executed by a different thread, sequential consistency of memory for all threads executing these callbacks is maintained.

  Maintaining memory consistency should make the api more developer friendly - especially for users of the validationinterface. To the extent that there are performance implications from this decision, these are not currently present in practice because all use of this scheduler happens on a single thread currently, furthermore the lock should guarantee consistency across callback executions even when callbacks are executed by multiple threads (as the test does).

Tree-SHA512: 5d95a7682c402e5ad76b05bc9dfbca99ca64105f62ab9e78f6fc0f6ea8c5277aa399fbb94298e35cc677b0c2181ff17259584bb7ae230e38aa68b85ecbc22856

Author: MarcoFalke

src/scheduler.h

@@ -86,9 +86,13 @@ class CScheduler
 
 /**
  * Class used by CScheduler clients which may schedule multiple jobs
- * which are required to be run serially. Does not require such jobs
- * to be executed on the same thread, but no two jobs will be executed
- * at the same time.
+ * which are required to be run serially. Jobs may not be run on the
+ * same thread, but no two jobs will be executed
+ * at the same time and memory will be release-acquire consistent
+ * (the scheduler will internally do an acquire before invoking a callback
+ * as well as a release at the end). In practice this means that a callback
+ * B() will be able to observe all of the effects of callback A() which executed
+ * before it.
  */
 class SingleThreadedSchedulerClient {
 private:
@@ -103,6 +107,13 @@ class SingleThreadedSchedulerClient {
 
 public:
     explicit SingleThreadedSchedulerClient(CScheduler *pschedulerIn) : m_pscheduler(pschedulerIn) {}
+
+    /**
+     * Add a callback to be executed. Callbacks are executed serially
+     * and memory is sequentially consistent between callback executions.
+     * Practially, this means that callbacks can behave as if they are executed
+     * in order by a single thread.
+     */
     void AddToProcessQueue(std::function<void (void)> func);
 
     // Processes all remaining queue members on the calling thread, blocking until queue is empty

src/test/scheduler_tests.cpp

@@ -65,7 +65,7 @@ BOOST_AUTO_TEST_CASE(manythreads)
     size_t nTasks = microTasks.getQueueInfo(first, last);
     BOOST_CHECK(nTasks == 0);
 
-    for (int i = 0; i < 100; i++) {
+    for (int i = 0; i < 100; ++i) {
         boost::chrono::system_clock::time_point t = now + boost::chrono::microseconds(randomMsec(rng));
         boost::chrono::system_clock::time_point tReschedule = now + boost::chrono::microseconds(500 + randomMsec(rng));
         int whichCounter = zeroToNine(rng);
@@ -112,4 +112,46 @@ BOOST_AUTO_TEST_CASE(manythreads)
     BOOST_CHECK_EQUAL(counterSum, 200);
 }
 
+BOOST_AUTO_TEST_CASE(singlethreadedscheduler_ordered)
+{
+    CScheduler scheduler;
+
+    // each queue should be well ordered with respect to itself but not other queues
+    SingleThreadedSchedulerClient queue1(&scheduler);
+    SingleThreadedSchedulerClient queue2(&scheduler);
+
+    // create more threads than queues
+    // if the queues only permit execution of one task at once then
+    // the extra threads should effectively be doing nothing
+    // if they don't we'll get out of order behaviour
+    boost::thread_group threads;
+    for (int i = 0; i < 5; ++i) {
+        threads.create_thread(boost::bind(&CScheduler::serviceQueue, &scheduler));
+    }
+
+    // these are not atomic, if SinglethreadedSchedulerClient prevents
+    // parallel execution at the queue level no synchronization should be required here
+    int counter1 = 0;
+    int counter2 = 0;
+
+    // just simply count up on each queue - if execution is properly ordered then
+    // the callbacks should run in exactly the order in which they were enqueued
+    for (int i = 0; i < 100; ++i) {
+        queue1.AddToProcessQueue([i, &counter1]() {
+            BOOST_CHECK_EQUAL(i, counter1++);
+        });
+
+        queue2.AddToProcessQueue([i, &counter2]() {
+            BOOST_CHECK_EQUAL(i, counter2++);
+        });
+    }
+
+    // finish up
+    scheduler.stop(true);
+    threads.join_all();
+
+    BOOST_CHECK_EQUAL(counter1, 100);
+    BOOST_CHECK_EQUAL(counter2, 100);
+}
+
 BOOST_AUTO_TEST_SUITE_END()

src/validationinterface.h

@@ -53,6 +53,21 @@ void CallFunctionInValidationInterfaceQueue(std::function<void ()> func);
  */
 void SyncWithValidationInterfaceQueue();
 
+/**
+ * Implement this to subscribe to events generated in validation
+ *
+ * Each CValidationInterface() subscriber will receive event callbacks
+ * in the order in which the events were generated by validation.
+ * Furthermore, each ValidationInterface() subscriber may assume that
+ * callbacks effectively run in a single thread with single-threaded
+ * memory consistency. That is, for a given ValidationInterface()
+ * instantiation, each callback will complete before the next one is
+ * invoked. This means, for example when a block is connected that the
+ * UpdatedBlockTip() callback may depend on an operation performed in
+ * the BlockConnected() callback without worrying about explicit
+ * synchronization. No ordering should be assumed across
+ * ValidationInterface() subscribers.
+ */
 class CValidationInterface {
 protected:
     /**