SPC: allow counters to be attached solely through MPI_T and reduce overhead

- only make MCA parameters available if SPC is enabled

- do not compile SPC code if SPC is disabled

- move includes into ompi_spc.c

- allow counters to be enabled through MPI_T without setting MCA parameter

- inline counter update calls that are likely in the critical path

- fix test to succeed even if encountering invalid pvars

- move timer_[start|stop] to header and move attachment info into ompi_spc_t

There is no need to store the name in the ompi_spc_t struct too, we can use that space
for the attachment info instead to avoid accessing another cache line.

- make timer/watermark flags a property of the spc description

This is meant to making adding counters easier in the future by
centralizing the necessary information. By storing a copy of these flags
in the ompi_spc_t structure (without adding to its size) reduces
cache pollution for timer/watermark events.

- allocate ompi_spc_t objects with cache-alignment

This prevents objects from spanning multiple cache lines and thus
ensures that only one cache line is loaded per update.

- fix handling of timer and timer conversion

- only call opal_timer_base_get_cycles if necesary to reduce overhead

- Remove use of OPAL_UNLIKELY to improve code generated by GCC

It appears that GCC makes less effort in optimizing the unlikely path
and generates bloated code.

- Allocate ompi_spc_events statically to reduce loads in critical path

- duplicate comm_world only when dumping is requested

Signed-off-by: Joseph Schuchart <[email protected]>

Author: devreal

ompi/runtime/ompi_mpi_params.c

@@ -334,21 +334,23 @@ int ompi_mpi_register_params(void)
                                  MCA_BASE_VAR_SCOPE_READONLY,
                                  &ompi_mpi_compat_mpi3);
 
+#if SPC_ENABLE == 1
     ompi_mpi_spc_attach_string = NULL;
     (void) mca_base_var_register("ompi", "mpi", NULL, "spc_attach",
-                                 "A comma delimeted string listing the software-based performance counters (SPCs) to enable.",
+                                 "A comma-delimeted list of software-based performance counters (SPCs) to enable (\"all\" enables all counters).",
                                  MCA_BASE_VAR_TYPE_STRING, NULL, 0, 0,
                                  OPAL_INFO_LVL_4,
                                  MCA_BASE_VAR_SCOPE_READONLY,
                                  &ompi_mpi_spc_attach_string);
 
     ompi_mpi_spc_dump_enabled = false;
     (void) mca_base_var_register("ompi", "mpi", NULL, "spc_dump_enabled",
-                                 "A boolean value for whether (true) or not (false) to enable dumping SPC counters in MPI_Finalize.",
+                                 "A boolean value for whether (true) or not (false) to enable dumping enabled SPC counters in MPI_Finalize.",
                                  MCA_BASE_VAR_TYPE_BOOL, NULL, 0, 0,
                                  OPAL_INFO_LVL_4,
                                  MCA_BASE_VAR_SCOPE_READONLY,
                                  &ompi_mpi_spc_dump_enabled);
+#endif // SPC_ENABLE
 
     return OMPI_SUCCESS;
 }