Merge pull request #85 from dzenanz/master

dzenanz · web-flow · commit 75530bdf94b2 · 2018-05-24T16:31:04.000-04:00
ENH: adding documentation for refactored multi-threading infrastructure
diff --git a/SoftwareGuide/Latex/Architecture/SystemOverview.tex b/SoftwareGuide/Latex/Architecture/SystemOverview.tex
@@ -18,7 +18,7 @@ \section{System Organization}
         concepts include generic programming, smart pointers for memory
         management, object factories for adaptable object instantiation,
         event management using the command/observer design paradigm, and
-        multithreading support.
+        multi-threading support.
 
   \item[Numerics.] ITK uses VXL's VNL numerics libraries. These are
         easy-to-use C++ wrappers around the Netlib Fortran numerical
@@ -36,7 +36,7 @@ \section{System Organization}
     \emph{filters} that in turn may be organized into data flow
     \emph{pipelines}. These pipelines maintain state and therefore
     execute only when necessary.  They also support
-    multithreading, and are streaming capable (i.e., can operate
+    multi-threading, and are streaming capable (i.e., can operate
     on pieces of data to minimize the memory footprint).
 
     \item[IO Framework.] Associated with the data processing
@@ -375,30 +375,100 @@ \subsection{Event Handling}
 \subsection{Multi-Threading}
 \label{sec:MultiThreading}
 
-Multithreading is handled in ITK through a high-level design
-abstraction. This approach provides portable multithreading and hides the
+Multi-threading is handled in ITK through a high-level design
+abstraction. This approach provides portable multi-threading and hides the
 complexity of differing thread implementations on the many systems supported
-by ITK. For example, the class \doxygen{MultiThreader} provides support for
-multithreaded execution using \code{sproc()} on an SGI, or
-\code{pthread\_create} on any platform supporting POSIX threads.
-
-Multithreading is typically employed by an algorithm during its execution
-phase. MultiThreader can be used to execute a single method on
-multiple threads, or to specify a method per thread. For example, in the
+by ITK. For example, the class \doxygen{PlatformMultiThreader} provides support for
+multi-threaded execution by directly using platform-specific primitives such as
+\code{pthread\_create}. \doxygen{TBBMultiThreader} uses Intel's
+Thread Building Blocks cross-platform library,
+which can do dynamic workload balancing across multiple processes.
+This means that \code{outputRegionForThread} might have different sizes
+which change over time, depending on overall processor load.
+All multi-threader implementations derive from \doxygen{MultiThreaderBase}.
+
+Multi-threading is typically employed by an algorithm during its execution
+phase. For example, in the
 class \doxygen{ImageSource} (a superclass for most image processing filters)
 the \code{GenerateData()} method uses the following methods:
 
 \small
 \begin{minted}[baselinestretch=1,fontsize=\footnotesize,linenos=false,bgcolor=ltgray]{cpp}
-  multiThreader->SetNumberOfThreads(int);
-  multiThreader->SetSingleMethod(ThreadFunctionType, void* data);
-  multiThreader->SingleMethodExecute();
+  this->GetMultiThreader()->template ParallelizeImageRegion<OutputImageDimension>(
+      this->GetOutput()->GetRequestedRegion(),
+      [this](const OutputImageRegionType & outputRegionForThread)
+        { this->DynamicThreadedGenerateData(outputRegionForThread); }, this);
+\end{minted}
+\normalsize
+
+In this example each thread invokes \code{DynamicThreadedGenerateData}
+method of the derived filter. The \code{ParallelizeImageRegion}
+method takes care to divide the image into different regions
+that do not overlap for write operations.
+\code{ImageSource}'s \code{GenerateData()} passes \code{this} pointer
+to \code{ParallelizeImageRegion}, which allows \code{ParallelizeImageRegion}
+to update the filter's progress after each region is finished processing.
+
+If a filter has some serial part in the middle, in addition to initialization
+done in \code{BeforeThreadedGenerateData()} and finalization done in
+\code{AfterThreadedGenerateData()}, it can parallelize more than one method
+in its own version of \code{GenerateData()}, such as done by
+\doxygen{CannyEdgeDetectionImageFilter}:
+
+\small
+\begin{minted}[baselinestretch=1,fontsize=\footnotesize,linenos=false,bgcolor=ltgray]{cpp}
+::GenerateData()
+{
+  this->UpdateProgress(0.0f);
+  Superclass::AllocateOutputs();
+  // Small serial section
+  this->UpdateProgress(0.01f);
+
+  // Calculate 2nd order directional derivative
+  this->GetMultiThreader()->template ParallelizeImageRegion<TOutputImage::ImageDimension>(
+      this->GetOutput()->GetRequestedRegion(),
+      [this](const OutputImageRegionType & outputRegionForThread)
+      { this->ThreadedCompute2ndDerivative(outputRegionForThread); }, nullptr);
+  this->UpdateProgress(0.45f);
+
+  // Calculate the gradient of the second derivative
+  this->GetMultiThreader()->template ParallelizeImageRegion<TOutputImage::ImageDimension>(
+      this->GetOutput()->GetRequestedRegion(),
+      [this](const OutputImageRegionType & outputRegionForThread)
+      { this->ThreadedCompute2ndDerivativePos(outputRegionForThread); }, nullptr);
+  this->UpdateProgress(0.9f);
+
+  // More processing
+  this->UpdateProgress(1.0f);
+}
 \end{minted}
 \normalsize
 
-In this example each thread invokes the same method. The multithreaded filter
-takes care to divide the image into different regions that do not overlap for
-write operations.
+When invoking \code{ParallelizeImageRegion} multiple times from
+\code{GenerateData()}, \code{nullptr} should be passed instead of \code{this},
+otherwise progress will go from 0\% to 100\% more than once. And this
+will at least confuse any other class watching the filter's progress events,
+even if it does not cause a crash. So the filter's author should estimate
+how long each part of \code{GenerateData()} takes, and update the progress
+``manually'' as in the example above.
+
+With ITK version 5.0, the Multi-Threading mechanism has been refactored.
+What was previously \code{itk::MultiThreader}, is now a hierarchy of classes.
+\doxygen{PlatformMultiThreader} is a slightly cleaned-up version of the old
+class - \code{MultipleMethodExecute} and \code{SpawnThread} methods have been
+deprecated. But much of its content has been moved to
+\doxygen{MultiThreaderBase}. And classes should use the multi-threaders via
+\code{MultiThreaderBase} interface, to allow the end user the flexibility to
+select the multi-threader at run time. This also allows the filter to
+benefit from future improvements in threading such as addition of a new
+multi-threader implementation.
+
+The backwards compatible \code{ThreadedGenerateData(Region, ThreadId)} method
+signature has been kept, for use in filters that must know their thread number.
+To use this signature, a filter must invoke
+\code{this->DynamicMultiThreadingOff();} before \code{Update();} is called by
+the filter's user or downstream filter in the pipeline. The best place for
+invoking \code{this->DynamicMultiThreadingOff();} is the filter's constructor.
 
 The general philosophy in ITK regarding thread safety is that accessing
 different instances of a class (and its methods) is a thread-safe operation.
@@ -501,7 +571,7 @@ \section{Data Representation}
 
 One of the important ITK concepts regarding images is that rectangular,
 continuous pieces of the image are known as \emph{regions}. Regions are used
-to specify which part of an image to process, for example in multithreading,
+to specify which part of an image to process, for example in multi-threading,
 or which part to hold in memory. In ITK there are three common types of
 regions:
 \begin{enumerate}
diff --git a/SoftwareGuide/Latex/DevelopmentGuidelines/WriteAFilter.tex b/SoftwareGuide/Latex/DevelopmentGuidelines/WriteAFilter.tex
@@ -6,7 +6,7 @@ \chapter{How To Write A Filter}
 parts. An initial definition of terms is followed by an overview of
 the filter creation process. Next, data streaming is discussed. The
 way data is streamed in ITK must be understood in order to write
-correct filters. Finally, a section on multithreading describes what
+correct filters. Finally, a section on multi-threading describes what
 you must do in order to take advantage of shared memory parallel
 processing.
 
@@ -160,15 +160,15 @@ \section{Overview of Filter Creation}
 parts of the implementation: defining the API, data members, and other
 implementation details of the algorithm. In particular, the filter writer
 will have to implement either a \code{GenerateData()} (non-threaded) or
-\code{ThreadedGenerateData()} method. (See Section~\ref{sec:MultiThreading}
-for an overview of multi-threading in ITK.)
+\code{ThreadedGenerateData()} and \code{DynamicThreadedGenerateData()} methods.
+(See Section~\ref{sec:MultiThreading} for an overview of multi-threading in ITK.)
 
 An important note: the GenerateData() method is required to allocate memory
 for the output. The ThreadedGenerateData() method is not. In default
 implementation (see \doxygen{ImageSource}, a superclass of
 \doxygen{ImageToImageFilter})
 \code{GenerateData()} allocates memory and then invokes
-\code{ThreadedGenerateData()}.
+\code{DynamicThreadedGenerateData()} or \code{ThreadedGenerateData()}.
 
 One of the most important decisions that the developer must make is whether
 the filter can stream data; that is, process just a portion of the input to
@@ -205,7 +205,7 @@ \section{Streaming Large Data}
 A significant benefit of this architecture is that the relatively complex
 process of managing pipeline execution is designed into the system. This
 means that keeping the pipeline up to date, executing only those portions of
-the pipeline that have changed, multithreading execution, managing memory
+the pipeline that have changed, multi-threading execution, managing memory
 allocation, and streaming is all built into the architecture. However, these
 features do introduce complexity into the system, the bulk of which is seen
 by class developers. The purpose of this chapter is to describe the pipeline
@@ -242,9 +242,9 @@ \subsection{Overview of Pipeline Execution}
         data processing kernels, that affect how much data input data
         (extra padding) is required.
 
-        \item It subdivides data into subpieces for multithreading. (Note
+        \item It subdivides data into subpieces for multi-threading. (Note
         that the division of data into subpieces is exactly same problem as
-        dividing data into pieces for streaming; hence multithreading comes
+        dividing data into pieces for streaming; hence multi-threading comes
         for free as part of the streaming architecture.)
 
         \item It may free (or release) output data if filters no longer need
@@ -444,7 +444,7 @@ \subsubsection{UpdateOutputData()}
 
 The developer will never override \code{UpdateOutputData()}. The developer need
 only write the \code{GenerateData()} method (non-threaded) or
-\code{ThreadedGenerateData()} method. A discussion of threading follows in the
+\code{DynamicThreadedGenerateData()} method. A discussion on threading follows in the
 next section.
 
 
@@ -454,29 +454,34 @@ \section{Threaded Filter Execution}
 
 Filters that can process data in pieces can typically multi-process
 using the data parallel, shared memory implementation built into the
-pipeline execution process. To create a multithreaded filter, simply
-define and implement a \code{ThreadedGenerateData()} method. For
-example, a \doxygen{ImageToImageFilter} would create the method:
+pipeline execution process. To create a multi-threaded filter, simply
+define and implement a \code{DynamicThreadedGenerateData()}.
+For example, a \doxygen{ImageToImageFilter} would create the method:
 
 \small
 \begin{minted}[baselinestretch=1,fontsize=\footnotesize,linenos=false,bgcolor=ltgray]{cpp}
-    void ThreadedGenerateData( const OutputImageRegionType&
-      outputRegionForThread, ThreadIdType threadId ) override;
+  void DynamicThreadedGenerateData( const OutputImageRegionType&
+    outputRegionForThread ) override;
 \end{minted}
 \normalsize
 
-The key to threading is to generate output for the output region given (as
-the first parameter in the argument list above). In ITK, this is simple to do
+The key to threading is to generate output for the output region given as
+the parameter. In ITK, this is simple to do
 because an output iterator can be created using the region provided. Hence
 the output can be iterated over, accessing the corresponding input pixels as
 necessary to compute the value of the output pixel.
 
 Multi-threading requires caution when performing I/O (including using
 \code{cout} or \code{cerr}) or invoking events. A safe practice is to allow
 only thread id zero to perform I/O or generate events. (The thread id is
-passed as argument into \code{ThreadedGenerateData()}).  If more than one
-thread tries to write to the same place at the same time, the program can
-behave badly, and possibly even deadlock or crash.
+passed as argument into \code{ThreadedGenerateData(Region, ThreadId)}).
+If more than one thread tries to write to the same place at the same time,
+the program can behave badly, and possibly even deadlock or crash.
+
+Filters which need thread identifier (id) should implement the
+\code{ThreadedGenerateData(Region, ThreadId)} method and call
+\code{this->DynamicMultiThreadingOff();} in the filter's constructor.
+
 
 
 \section{Filter Conventions}
