Reduction binary ops on floating-points (and NaN) might not be strictly associative/commutative

[davidozog]

Problem Statement

From the OpenSHMEM v1.6 project, we encountered this issue:
OpenSHMEM 1.6 Section Committees (view)

Reduction binary ops on floating-points (and NaN) might not be strictly associative/commutative.

Proposed Changes/Ideas

• The result should be the same on all PEs.
• The result should be the same from run to run.
• The result may be different on different architectures, due to differing arithmetic.
• The result on IEEE-754 compatible platforms should always be the same everywhere every time.
• We might consider new APIs or environment variables that enforce more consistent results.
• A team could take an optional config parameter indicating desire for a well-ordered reduction (i.e, C++ std::reduce vs std::accumulate)

References and Pull Requests

(original v1.6 project issue) OpenSHMEM 1.6 Section Committees (view)
(closed) davidozog#14
davidozog#16