Wrote regarding better speedups for all of those functions in terms of having more rows vs having more columns after some testing (benchmarking) and research

Author: Anirban166

man/openmp-utils.Rd

@@ -45,6 +45,8 @@
       \item The checking and handling of undefined values (such as NaNs)
     }
     
+    Since this function is used to find rows where a column's value falls within a specific range, it benefits more from parallelization when the input data consists of a large number of rows.
+    
     \item\file{cj.c} - \code{\link{CJ}()}
 
     OpenMP is used here to parallelize:
@@ -55,6 +57,8 @@
       \item The creation of all combinations of the input vectors over the cross-product space
     }
 
+    Given that the number of combinations increases exponentially as more columns are added, better speedup can be expected when dealing with a large number of columns.
+    
     \item\file{coalesce.c} - \code{\link{fcoalesce}()}
 
     OpenMP is used here to parallelize:
@@ -64,10 +68,14 @@
       \item The conditional checks within parallelized loops
       }
 
+    Significant speedup can be expected for more number of columns here, given that this function operates efficiently across multiple columns to find non-NA values.
+    
     \item\file{fifelse.c} - \code{\link{fifelse}()}
 
     For logical, integer, and real types, OpenMP is being used here to parallelize loops that perform conditional checks along with assignment operations over the elements of the supplied logical vector based on the condition (\code{test}) and values provided for the remaining arguments (\code{yes}, \code{no}, and \code{na}).
 
+    Better speedup can be expected for more number of columns here as well, given that this function operates column-wise with independent vector operations.
+    
     \item\file{fread.c} - \code{\link{fread}()}
 
     OpenMP is used here to:
@@ -78,6 +86,8 @@
       }
     There are no explicit pragmas for parallelizing loops, and instead the use of OpenMP here is mainly in controlling access to shared resources (with the use of critical sections, for instance) in a multi-threaded environment.
 
+    This function is highly optimized in reading and processing data with both large numbers of rows and columns, but the efficiency is more pronounced across rows.
+    
     \item\file{forder.c}, \file{fsort.c}, and \file{reorder.c} - \code{\link{forder}()} and related
 
     OpenMP is used here to parallelize multiple operations that come together to sort a \code{data.table} using the Radix algorithm. These include:
@@ -88,26 +98,38 @@
       \item Creation of histograms which are used to sort data based on significant bits (each thread processes a separate batch of the data, computes the MSB of each element, and then increments the corresponding bins), with the distribution and merging of buckets (specific to \file{fsort.c})
       \item The process of reordering a vector or each column in a list of vectors (such as in a \code{data.table}) based on a given vector that dictates the new ordering of elements (specific to \file{reorder.c})
       }
+
+    Better speedups can be expected when the input data contains a large number of rows as the sorting complexity increases with more rows.
 
     \item\file{froll.c}, \file{frolladaptive.c}, and \file{frollR.c} - \code{\link{froll}()} and family
 
     OpenMP is used here to parallelize the loops that compute the rolling means (\code{frollmean}) and sums (\code{frollsum}) over a sliding window for each position in the input vector.
+    
+    These functions benefit more in terms of speedup when the data has a large number of columns, primarily due to the efficient memory access patterns (cache-friendly) used when processing the data for each column sequentially in memory to compute the rolling statistic.
 
     \item\file{fwrite.c} - \code{\link{fwrite}()}
 
     OpenMP is used here primarily to parallelize the process of writing rows to the output file, but error handling and compression (if enabled) are also managed within the parallel region. Special attention is paid to thread safety and synchronization, especially in the ordered sections where output to the file and handling of errors is serialized to maintain the correct sequence of rows.
 
+    Similar to \code{\link{fread}()}, this function is highly efficient in parallely processing data with large numbers of both rows and columns, but it has more notable speedups with an increased number of rows.
+    
     \item\file{gsumm.c} - GForce in various places, see \link{GForce}
 
-    Functions with GForce optimization are internally parallelized to speed up grouped summaries over a large \code{data.table}. OpenMP is used here to parallelize operations involved in calculating group-wise statistics like sum, mean, and median (implying faster computation of \code{sd}, \code{var}, and \code{prod} as well). The input data is split into batches (groups), and each thread processes a subset of the data based on them.
+    Functions with GForce optimization are internally parallelized to speed up grouped summaries over a large \code{data.table}. OpenMP is used here to parallelize operations involved in calculating group-wise statistics like sum, mean, and median (implying faster computation of \code{sd}, \code{var}, and \code{prod} as well). 
+    
+    These optimized grouping operations benefit more in terms of speedup if the input data contains a large number of rows (since they are often used to aggregate data across groups).
 
     \item\file{nafill.c} - \code{\link{nafill}()}
 
-    OpenMP is being used here to parallelize the loop that fills missing values over columns of the input data. This includes handling different data types (double, integer, and integer64) and applying the designated filling method (constant, last observation carried forward, or next observation carried backward) to each column in parallel.
+    OpenMP is being used here to parallelize the loop that fills missing values over columns of the input data. This includes handling different data types (double, integer, and integer64) and applying the designated filling method (constant, last observation carried forward, or next observation carried backward) to each column in parallel. 
+    
+    Given its optimization for column-wise operations, better speedups can be expected when the input data consists of a large number of columns.
 
     \item\file{subset.c} - Used in \code{\link[=data.table]{[.data.table}} subsetting
 
-    OpenMP is used here to parallelize the loops that perform the subsetting of vectors, with conditional checks and filtering of data.
+    OpenMP is used here to parallelize the loops that perform the subsetting of vectors, with conditional checks and filtering of data. 
+    
+    Since subset operations tend to be usually row-dependent, better speedups can be expected when dealing with a large number of rows. However, it also depends on whether the computations are focused on rows or columns (as dictated by the subsetting criteria).
 
     \item\file{types.c} - Internal testing usage