renamed to floating-point-rounding-errors

Author: Your Name

content/learning-paths/cross-platform/floating-point-rounding-errors/_index.md

@@ -0,0 +1,42 @@
+---
+title: Learn about floating point rounding errors on Arm and x86
+
+minutes_to_complete: 30
+
+who_is_this_for: Developers porting applications from x86 to AArch64 who observe different results on each platform.
+
+learning_objectives: 
+    - Understand the differences between converting floating point numbers on x86 and Arm. 
+    - Understand factors that affect floating point behaviour
+    - How to use basic compiler flags to produce predictable behaviour
+
+prerequisites:
+    - Access to an x86 and Arm-based machine
+    - Basic understanding of floating point numbers
+    - A C++/C compiler
+
+author: Kieran Hejmadi
+
+### Tags
+skilllevels: Introductory
+subjects: Performance and Architecture
+armips:
+    - Cortex-A
+    - Neoverse
+tools_software_languages:
+    - C++
+
+further_reading:
+    - resource:
+        title: G++ Optimisation Flags 
+        link: https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
+        type: documentation
+
+
+
+### FIXED, DO NOT MODIFY
+# ================================================================================
+weight: 1                       # _index.md always has weight of 1 to order correctly
+layout: "learningpathall"       # All files under learning paths have this same wrapper
+learning_path_main_page: "yes"  # This should be surfaced when looking for related content. Only set for _index.md of learning path content.
+---

content/learning-paths/cross-platform/floating-point-rounding-errors/_next-steps.md

@@ -0,0 +1,8 @@
+---
+# ================================================================================
+#       FIXED, DO NOT MODIFY THIS FILE
+# ================================================================================
+weight: 21                  # Set to always be larger than the content in this path to be at the end of the navigation.
+title: "Next Steps"         # Always the same, html page title.
+layout: "learningpathall"   # All files under learning paths have this same wrapper for Hugo processing.
+---

content/learning-paths/cross-platform/floating-point-rounding-errors/differences.png

@@ -0,0 +1,37 @@
+---
+title: Floating Point Representations
+weight: 2
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## Recap on Floating Point Numbers
+
+If you are unfamiliar with floating point representations, we recommend looking at this [introductory learning path](https://learn.arm.com/learning-paths/cross-platform/integer-vs-floats/introduction-integer-float-types/). 
+
+As a recap, floating-point numbers are a fundamental representation of real numbers in computer systems, enabling efficient storage and computation of decimal values with varying degrees of precision. In C/C++, floating point variables are created with keywords such as  `float` or `double`. The IEEE 754 standard, established in 1985, is the most widely used format for floating-point arithmetic, ensuring consistency across different hardware and software implementations.
+
+IEEE 754 defines two primary formats: single-precision (32-bit) and double-precision (64-bit). Each floating-point number consists of three components: 
+- **sign bit**. (Determining positive or negative value)
+- **exponent** (defining the scale or magnitude)
+- **significand** (also called the mantissa, representing the significant digits of the number). 
+
+The standard uses a biased exponent to handle both large and small numbers efficiently, and it incorporates special values such as NaN (Not a Number), infinity, and subnormal numbers for robust numerical computation. A key feature of IEEE 754 is its support for rounding modes and exception handling, ensuring predictable behavior in mathematical operations. However, floating-point arithmetic is inherently imprecise due to limited precision, leading to small rounding errors.
+
+The graphic below illustrates various forms of floating point representation supported by Arm, each with varying number of bits assigned to the exponent and matissa.
+
+![floating-point](./floating-point-numbers.png)
+
+## Rounding Errors 
+
+As mentioned above, since we are using a finite number of bits to store a continuous range of numbers, we introduce rounding error. The unit in last place (ULP) is the smallest difference between two consecutive floating-point numbers. It measures floating-point rounding error, which arises because not all real numbers can be exactly represented. When an operation is performed, the result is rounded to the nearest representable value, introducing a small error. This error, often measured in ULPs, indicates how close the computed value is to the exact result. For a simple example, if we construct a floating-point schema with 3 bits for the mantissa (precision) and an exponent in the range of -1 to 2. The possible values will look like the graph below. 
+
+![ulp](./ulp.png)
+
+Key takeaways:
+
+- ULP size varies with the number’s magnitude.
+- Larger numbers have bigger ULPs due to wider spacing between values.
+- Smaller numbers have smaller ULPs, reducing quantization error.
+- ULP behavior impacts numerical stability and precision in computations.

content/learning-paths/cross-platform/floating-point-rounding-errors/floating-point-numbers.png

@@ -0,0 +1,108 @@
+---
+title: Differences between x86 and Arm
+weight: 3
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## Differences in behaviour between x86 and Arm. 
+
+Architecture and standards define dealing with floating point overflows and truncations in difference ways. First, connect to a x86 and Arm-based machine. In this example I am connecting to an AWS `t2.2xlarge` and `t4g.xlarge` running Ubuntu 22.04 LTS. 
+
+
+To demonstrate this, the C++ code snippet below casts floating point numbers to various data types. Copy and paste into a new file called `converting-float.cpp`. 
+
+```cpp
+#include <iostream>
+#include <cmath>
+#include <limits>
+#include <cstdint>
+
+void convertFloatToInt(float value) {
+    // Convert to unsigned 32-bit integer
+    uint32_t u32 = static_cast<uint32_t>(value);
+
+    // Convert to signed 32-bit integer
+    int32_t s32 = static_cast<int32_t>(value);
+
+    // Convert to unsigned 16-bit integer (truncation happens)
+    uint16_t u16 = static_cast<uint16_t>(u32); 
+    uint8_t u8 = static_cast<uint8_t>(value); 
+
+    // Convert to signed 16-bit integer (truncation happens)
+    int16_t s16 = static_cast<int16_t>(s32);
+
+    std::cout << "Floating-Point Value: " << value << "\n";
+    std::cout << "  → uint32_t:  " << u32 << " (0x" << std::hex << u32 << std::dec << ")\n";
+    std::cout << "  → int32_t:   " << s32 << " (0x" << std::hex << s32 << std::dec << ")\n";
+    std::cout << "  → uint16_t (truncated):  " << u16 << " (0x" << std::hex << u16 << std::dec << ")\n";
+    std::cout << "  → int16_t (truncated):   " << s16 << " (0x" << std::hex << s16 << std::dec << ")\n";
+    std::cout << "  → uint8_t (truncated):   " << static_cast<int>(u8) << std::endl;
+
+    std::cout << "----------------------------------\n";
+}
+
+int main() {
+    std::cout << "Demonstrating Floating-Point to Integer Conversion\n\n";
+
+    // Test cases
+    convertFloatToInt(42.7f);                   // Normal case
+    convertFloatToInt(-15.3f);                  // Negative value -> wraps on unsigned
+    convertFloatToInt(4294967296.0f);           // Overflow: 2^32 (UINT32_MAX + 1)
+    convertFloatToInt(3.4e+38f);                // Large float exceeding UINT32_MAX
+    convertFloatToInt(-3.4e+38f);               // Large negative float
+    convertFloatToInt(NAN);                     // NaN behavior on different platforms
+    return 0;
+}
+```
+
+To demonstrate we will compile `converting-float.cpp` on an Arm64 and x86 machine. Install the `g++` compiler with the following command. I am using `G++` version 13.3 at the time of writing. 
+
+```bash
+sudo apt update
+sudo apt install g++ gcc
+```
+
+
+ Run the command below on both an Arm-based and x86-based system. 
+
+```bash
+g++ converting-float.cpp -o converting-float 
+```
+
+For easy comparison, the image below shows the x86 output (left) and Arm output (right). The  highlighted lines show the difference in output. 
+
+![differences](./differences.png)
+
+As you can see, there are several cases where different behaviour is observed. For example when trying to convert a signed number to a unsigned number or dealing with out-of-bounds numbers. 
+
+## Removing Hardcoded values with Macros
+
+Clearly the above differences show that explictly checking for specific values will lead to unportable code. For example, consider the naively implemented function below. This checks if the value is 0, the value an x86 machine will convert a floating point number that exceeds the maximum 32-bit float value. This is different to AArch64 behaviour leading to unportable code. 
+
+```cpp
+void checkFloatToUint32(float num) {
+    uint32_t castedNum = static_cast<uint32_t>(num);
+    if (castedNum == 0) {
+        std::cout << "The casted number is 0, indicating the float could out of bounds for uint32_t." << std::endl;
+    } else {
+        std::cout << "The casted number is: " << castedNum << std::endl;
+    }
+}
+```
+
+This can simply be corrected by using the macro, `UINT32_MAX`. 
+{{% notice Note %}} To find out all the available compiler-defined macros, the `echo | <clang|gcc etc.> -dM -` will output them to the terminal{{% /notice %}}
+
+```cpp
+void checkFloatToUint32(float num) {
+    uint32_t castedNum = static_cast<uint32_t>(num);
+    if (castedNum == UINT32_MAX) {
+        std::cout << "The casted number is " << UINT32_MAX <<  " indicating the float was out of bounds for uint32_t." << std::endl;
+    } else {
+        std::cout << "The casted number is: " << castedNum << std::endl;
+    }
+}
+```
+

content/learning-paths/cross-platform/floating-point-rounding-errors/how-to-1.md

@@ -0,0 +1,74 @@
+---
+title: Example
+weight: 3
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## Error Propagation
+
+One cause of different outputs between x86 and Arm stems from the order of instructions and how errors are propagated. As a hypothetical example, an Arm architecture may decide to reorder the instructions that each has a different rounding error (described in the unit in last place section) so that subtle changes are observed. Alternatively, 2 functions that are mathematically equivalent will propagate errors differently on a computer. 
+
+Consider the example below. Function 1, `f1` and 2, `f2` are mathematically equivalent. Hence they should return the same value given the same input. If we input a very small number, `1e-8`, the error is different due to the loss in precision caused by different operations. Specifically, function `f2` avoids the subtraction of nearly equal number. The full reasoning is out of scope but for those interested should look into the topic of [numerical stability](https://en.wikipedia.org/wiki/Numerical_stability). 
+
+Copy and paste the C++ snippet below into a file named `error-propagation.cpp`. 
+
+
+```cpp
+#include <stdio.h>
+#include <math.h>
+
+// Function 1: Computes sqrt(1 + x) - 1 using the naive approach
+float f1(float x) {
+    return sqrtf(1 + x) - 1;
+}
+
+// Function 2: Computes the same value using an algebraically equivalent transformation
+// This version is numerically more stable
+float f2(float x) {
+    return x / (sqrtf(1 + x) + 1);
+}
+
+int main() {
+    float x = 1e-8;  // A small value that causes floating-point precision issues
+    float result1 = f1(x);
+    float result2 = f2(x);
+
+    // Theoretically, result1 and result2 should be the same
+    float difference = result1 - result2;
+    // Multiply by a large number to amplify the error
+    float final_result = 100000000.0f * difference + 0.0001f;
+
+    // Print the results
+    printf("f1(%e) = %.10f\n", x, result1);
+    printf("f2(%e) = %.10f\n", x, result2);
+    printf("Difference (f1 - f2) = %.10e\n", difference);
+    printf("Final result after magnification: %.10f\n", final_result);
+
+    return 0;
+}
+```
+
+Compile the source code on both x86 and Arm64 with the following command.
+
+```bash
+g++ -g error-propagation.cpp -o error-propagation
+```
+
+Running the 2 binaries shows that the second function, f2, has a small rounding error on both architectures. Additionally, there is a further rounding difference when run on x86 compared to Arm.
+
+on x86:
+```output
+f1(1.000000e-08) = 0.0000000000
+f2(1.000000e-08) = 0.0000000050
+Difference (f1 - f2) = -4.9999999696e-09
+Final result after magnification: -0.4999000132
+```
+on Arm:
+```output
+f1(1.000000e-08) = 0.0000000000
+f2(1.000000e-08) = 0.0000000050
+Difference (f1 - f2) = -4.9999999696e-09
+Final result after magnification: -0.4998999834
+```

content/learning-paths/cross-platform/floating-point-rounding-errors/how-to-2.md

@@ -0,0 +1,79 @@
+---
+title: Minimising Variability across platforms
+weight: 3
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## Minimising Variability across platforms
+
+The line `#pragma STDC FENV_ACCESS ON` is a directive that informs the compiler to enable access to the floating-point environment. This is part of the C++11 standard and is used to ensure that the program can properly handle floating-point exceptions and rounding modes enabling your program to continue running if an exception is thrown. For more information, refer to the [documentation in the C++11 standard](https://en.cppreference.com/w/cpp/numeric/fenv).
+
+In the context below, enabling floating-point environment access is crucial because the functions you are working with involve floating-point arithmetic, which can be prone to precision errors and exceptions such as overflow, underflow, division by zero, and invalid operations. However, in our example since we are hardcoding the inputs this is not strictly necessary but is included as it may be relevant for your own application. 
+
+This directive is particularly important when performing operations that require high numerical stability and precision, such as the square root calculations in functions below. It allows the program to manage the floating-point state and handle any anomalies that might occur during these calculations, thereby improving the robustness and reliability of your numerical computations.
+
+Save the C++ file below as `error-propagation-min.cpp`. 
+
+```cpp
+#include <cstdio>
+#include <cmath>
+#include <cfenv>
+
+// Enable floating-point exceptions
+#pragma STDC FENV_ACCESS ON
+
+// Function 1: Computes sqrt(1 + x) - 1 using the naive approach
+double f1(double x) {
+    return sqrt(1 + x) - 1;
+}
+
+// Function 2: Computes the same value using an algebraically equivalent transformation
+// This version is numerically more stable
+double f2(double x) {
+    return x / (sqrt(1 + x) + 1);
+}
+
+int main() {
+    // Enable all floating-point exceptions
+    std::feclearexcept(FE_ALL_EXCEPT);
+    std::feraiseexcept(FE_DIVBYZERO | FE_INVALID | FE_OVERFLOW);
+
+    double x = 1e-8;  // A small value that causes floating-point precision issues
+    double result1 = f1(x);
+    double result2 = f2(x);
+
+    // Theoretically, result1 and result2 should be the same
+    double difference = result1 - result2;
+    // Multiply by a large number to amplify the error
+    double final_result = 100000000.0 * difference + 0.0001;
+
+    // Print the results
+    printf("f1(%e) = %.10f\n", x, result1);
+    printf("f2(%e) = %.10f\n", x, result2);
+    printf("Difference (f1 - f2) = %.10e\n", difference);
+    printf("Final result after magnification: %.10f\n", final_result);
+
+    return 0;
+}
+```
+
+Compile with the following command. In addition, we pass the C++ flag, `-frounding-math`. You should use use when your program dynamically changes the floating-point rounding mode or needs to run correctly under different rounding modes. In our example, it results in a predictable rounding mode on function `f1` across x86 and Arm64. For more information, please refer to the [G++ documentation](https://gcc.gnu.org/onlinedocs/gcc-13.3.0/gcc/Optimize-Options.html)
+
+```bash
+g++ -o error-propagation-min error-propagation-min.cpp -frounding-math
+```
+
+Running the new binary on both platforms leads to function, `f1` having a similar value to `f2`. Further the difference is now identical across both Arm64 and x86. 
+
+```output
+./error-propagation-min 
+f1(1.000000e-08) = 0.0000000050
+f2(1.000000e-08) = 0.0000000050
+Difference (f1 - f2) = -1.7887354748e-17
+Final result after magnification: 0.0000999982
+```
+
+{{% notice Note %}} G++ provides several compiler flags to help balance accuracy and performance such as`-ffp-contract` which is useful when lossy, fused operations are used, for example, fused-multiple. As another example `-ffloat-store` which prevent floating point variables from being stored in registers which can have different levels of precision and rounding. **Please refer to your compiler documentation for more information on the available flags**{{% /notice %}}
+