Merge pull request #1994 from madeline-underwood/cache

false_sharing_JA to review

Author: jasonrandrews

content/learning-paths/servers-and-cloud-computing/false-sharing-arm-spe/_index.md

@@ -1,16 +1,12 @@
 ---
-title: Analyze cache behavior with Perf C2C on Arm
-
-draft: true
-cascade:
-    draft: true
+title: Analyze cache behavior with perf c2c on Arm
 
 minutes_to_complete: 15
 
-who_is_this_for: This topic is for developers who want to optimize cache access patterns on Arm servers using Perf C2C. 
+who_is_this_for: This topic is for performance-oriented developers working on Arm-based cloud or server systems who want to optimize memory access patterns and investigate cache inefficiencies using Perf C2C and Arm SPE.
 
 learning_objectives: 
-    - Avoid false sharing in C++ using memory alignment.
+    - Identify and fix false sharing issues using Perf C2C, a cache line analysis tool.
     - Enable and use the Arm Statistical Profiling Extension (SPE) on Linux systems.
     - Investigate cache line performance with Perf C2C.
 

content/learning-paths/servers-and-cloud-computing/false-sharing-arm-spe/how-to-1.md

@@ -1,36 +1,61 @@
 ---
-title: Introduction to Arm SPE and false sharing
+title: Arm Statistical Profiling Extension and false sharing
 weight: 2
 
 ### FIXED, DO NOT MODIFY
 layout: learningpathall
 ---
 
-## Introduction to the Arm Statistical Profiling Extension (SPE)
+## What is the Arm Statistical Profiling Extension (SPE), and what does it do?
 
-Standard performance tracing relies on counting completed instructions, capturing only architectural instructions without revealing the actual memory addresses, pipeline latencies, or considering micro-operations in flight. Moreover, the “skid” phenomenon where events are falsely attributed to later instructions can mislead developers. 
+{{% notice Learning goal%}}
+In this section, you’ll learn how to use SPE to gain low-level insight into how your applications interact with the CPU. You’ll explore how to detect and resolve false sharing. By combining cache line alignment techniques with `perf c2c`, you can identify inefficient memory access patterns and significantly boost CPU performance on Arm-based systems.
+{{% /notice %}}
 
-SPE integrates sampling directly into the CPU pipeline, triggering on individual micro-operations rather than retired instructions, thereby eliminating skid and blind spots. Each SPE sample record includes relevant metadata, such as data addresses, per-µop pipeline latency, triggered PMU event masks, and the memory hierarchy source, enabling fine-grained and precise cache analysis. 
+Arm’s Statistical Profiling Extension (SPE) gives you a powerful way to understand what’s really happening inside your applications at the microarchitecture level. 
 
-This enables software developers to tune user-space software for characteristics such as memory latency and cache accesses. Importantly, cache statistics are enabled with the Linux Perf cache-to-cache (C2C) utility.
+Introduced in Armv8.2, SPE captures a statistical view of how instructions move through the CPU, which allows you to dig into issues like memory access latency, cache misses, and pipeline behavior.
 
-Please refer to the [Arm SPE white paper](https://developer.arm.com/documentation/109429/latest/) for more details. 
+Most Linux profiling tools focus on retired instruction counts, which means they miss key details like memory addresses, cache latency, and micro-operation behavior. This can lead to misleading results, especially due to a phenomenon called “skid,” where events are falsely attributed to later instructions.
 
-In this Learning Path, you will use SPE and Perf C2C to diagnose a cache issue for an application running on a Neoverse server.
+SPE integrates sampling directly into the CPU pipeline, triggering on individual micro-operations instead of retired instructions. This approach eliminates skid and blind spots. Each SPE sample record includes relevant metadata, such as:
 
-## False sharing within the cache
+* Data addresses
+* Per-µop pipeline latency
+* Triggered PMU event masks
+* Memory hierarchy source
 
-Even when two threads touch entirely separate variables, modern processors move data in fixed-size cache lines (nominally 64-bytes). If those distinct variables happen to occupy bytes within the same line, every time one thread writes its variable the core’s cache must gain exclusive ownership of the whole line, forcing the other core’s copy to be invalidated. The second thread, still working on its own variable, then triggers a coherence miss to fetch the line back, and the ping-pong pattern repeats. Please see the illustration below, taken from the Arm SPE white paper, for a visual explanation.
+This enables fine-grained, precise cache analysis. 
 
-![false_sharing_diagram](./false_sharing_diagram.png)
+SPE helps developers optimize user-space applications by showing where cache latency or memory access delays are happening. Importantly, cache statistics are enabled with the Linux `perf` cache-to-cache (C2C) utility.
 
-Because false sharing hides behind ordinary writes, the easiest time to eliminate it is while reading or refactoring the source code by padding or realigning the offending variables before compilation. In large, highly concurrent codebases, however, data structures are often accessed through several layers of abstraction, and many threads touch memory via indirection, so the subtle cache-line overlap may not surface until profiling or performance counters reveal unexpected coherence misses.
+For more information, see the [*Arm Statistical Profiling Extension: Performance Analysis Methodology White Paper*](https://developer.arm.com/documentation/109429/latest/). 
+
+In this Learning Path, you will use SPE and `perf c2c` to diagnose a cache issue for an application running on a Neoverse server.
+
+## What is false sharing and why should I care about it?
+
+In large-scale, multithreaded applications, false sharing can degrade performance by introducing hundreds of unnecessary cache line invalidations per second - often with no visible red flags in the source code.
+
+Even when two threads touch entirely separate variables, modern processors move data in fixed-size cache lines, which is typically 64 bytes. If those distinct variables happen to occupy bytes within the same line, every time one thread writes its variable the core’s cache must gain exclusive ownership of the whole line, forcing the other core’s copy to be invalidated. 
+
+The second thread, still working on its own variable, then triggers a coherence miss to fetch the line back, and the ping-pong pattern repeats. 
+
+The diagram below, taken from the Arm SPE white paper, provides a visual representation of two threads on separate cores alternately gaining exclusive access to the same cache line.
+
+![false_sharing_diagram alt-text#center](./false_sharing_diagram.png "Two threads on separate cores alternately gain exclusive access to the same cache line.")
+
+## Why false sharing is hard to spot and fix
+
+False sharing often hides behind seemingly ordinary writes, making it tricky to catch without tooling. The best time to eliminate it is early, while reading or refactoring code, by padding or realigning variables before compilation. But in large, highly concurrent C++ codebases, memory is frequently accessed through multiple layers of abstraction. Threads may interact with shared data indirectly, causing subtle cache line overlaps that don’t become obvious until performance profiling reveals unexpected coherence misses. Tools like `perf c2c` can help uncover these issues by tracing cache-to-cache transfers and identifying hot memory locations affected by false sharing.
 
 From a source-code perspective nothing is “shared,” but at the hardware level both variables are implicitly coupled by their physical location.
 
 ## Alignment to cache lines
 
-In C++11, you can manually specify the alignment of an object with the `alignas` specifier. For example, the C++11 source code below manually aligns the the `struct` every 64 bytes (typical cache line size on a modern processor). This ensures that each instance of `AlignedType` is on a separate cache line. 
+In C++11, you can manually specify the alignment of an object with the `alignas` specifier. 
+
+For example, the C++11 source code below manually aligns the `struct` every 64 bytes (typical cache line size on a modern processor). This ensures that each instance of `AlignedType` is on a separate cache line. 
 
 ```cpp
 #include <atomic>
@@ -43,7 +68,7 @@ struct alignas(64) AlignedType {
 
 
 int main() {
-  // If we create four atomic integers like this, there's a high probability
+  // If you create four atomic integers like this, there's a high probability
   // they'll wind up next to each other in memory
   std::atomic<int> a;
   std::atomic<int> b;
@@ -74,9 +99,9 @@ int main() {
 }
 ```
 
-The example output below shows the variables e, f, g and h occur at least 64-bytes apart in the byte-addressable architecture. Whereas variables a, b, c and d occur 8 bytes apart, occupying the same cache line. 
+The output below shows that the variables e, f, g and h occur at least 64 bytes apart in the byte-addressable architecture. Whereas variables a, b, c, and d occur 8 bytes apart, occupying the same cache line. 
 
-Although this is a contrived example, in a production workload there may be several layers of indirection that unintentionally result in false sharing. For these complex cases, to understand the root cause you will use Perf C2C.
+Although this is a simplified example, in a production workload there might be several layers of indirection that unintentionally result in false sharing. For these complex cases, use `perf c2c` to trace cache line interactions and pinpoint the root cause of performance issues.
 
 ```output
 Without Alignment can occupy same cache line
@@ -96,4 +121,10 @@ Address of AlignedType g - 0xffffeb6c60c0
 Address of AlignedType h - 0xffffeb6c6080
 ```
 
-Continue to the next section to learn how to set up a system to run Perf C2C.
+## Summary
+
+In this section, you explored what Arm SPE is and why it offers a deeper, more accurate view of application performance. You also examined how a subtle issue like false sharing can impact multithreaded code, and how to mitigate it using data alignment techniques in C++.
+
+Next, you'll set up your environment and use `perf c2c` to capture and analyze real-world cache behavior on an Arm Neoverse system.
+
+

content/learning-paths/servers-and-cloud-computing/false-sharing-arm-spe/how-to-2.md

@@ -1,20 +1,23 @@
 ---
-title: Configure your environment for Arm SPE profiling
+title: Set up your environment for Arm SPE and perf c2c profiling
 weight: 3
 
 ### FIXED, DO NOT MODIFY
 layout: learningpathall
 ---
-
 ## Select a system with SPE support
 
-SPE requires both hardware and operating system support. Many cloud instances running Linux do not enable SPE-based profiling.
+{{% notice Learning goal%}}
+Before you can start profiling cache behavior with Arm SPE and `perf c2c`, your system needs to meet a few requirements. In this section, you’ll learn how to check whether your hardware and kernel support Arm SPE, install the necessary tools, and validate that Linux perf can access the correct performance monitoring events. By the end, your environment will be ready to record and analyze memory access patterns using `perf c2c` on an Arm Neoverse system.
+{{% /notice %}}
+
+SPE requires support from both your hardware and the operating system. Many cloud instances running Linux do not enable SPE-based profiling.
 
 You need to identify a system that supports SPE using the information below. 
 
 If you are looking for an AWS system, you can use a `c6g.metal` instance running Amazon Linux 2023 (AL2023). 
 
-Check the underlying Neoverse processor and operating system kernel version with the following commands. 
+Check the underlying Neoverse processor and operating system kernel version with the following commands: 
 
 ```bash
 lscpu | grep -i "model name"
@@ -23,7 +26,7 @@ uname -r
 
 The output includes the CPU type and kernel release version:
 
-```ouput
+```output
 Model name:                           Neoverse-N1
 6.1.134-152.225.amzn2023.aarch64
 ```
@@ -35,23 +38,23 @@ sudo dnf update -y
 sudo dnf install perf git gcc cmake numactl-devel -y
 ```
 
-Linux Perf is a userspace process and SPE is a hardware feature. The Linux kernel must be compiled with SPE support or the kernel module named `arm_spe_pmu` must be loaded.
+Linux perf is a userspace process and SPE is a hardware feature. The Linux kernel must be compiled with SPE support or the kernel module named `arm_spe_pmu` must be loaded.
 
 Run the following command to confirm if the SPE kernel module is loaded:
 
 ```bash
 sudo modprobe arm_spe_pmu
 ```
 
-If the module is not loaded (blank output), SPE may still be available.
+If the module is not loaded (and there is blank output), SPE might still be available.
 
 Run this command to check if SPE is included in the kernel:
 
 ```bash
 ls /sys/bus/event_source/devices/ | grep arm_spe
 ```
 
-If SPE is available, the output is:
+If SPE is available, the output you will see is:
 
 ```output
 arm_spe_0
@@ -63,11 +66,11 @@ If the output is blank then SPE is not available.
 
 You can install and run a Python script named Sysreport to summarize your system's performance profiling capabilities.
 
-Refer to [Get ready for performance analysis with Sysreport](https://learn.arm.com/learning-paths/servers-and-cloud-computing/sysreport/) to learn how to install and run it.
+See the Learning Path [Get ready for performance analysis with Sysreport](https://learn.arm.com/learning-paths/servers-and-cloud-computing/sysreport/) to learn how to install and run it.
 
 Look at the Sysreport output and confirm SPE is available by checking the `perf sampling` field. 
 
-If the printed value is SPE then SPE is available.
+If the printed value is SPE, then SPE is available.
 
 ```output
 ...
@@ -83,9 +86,9 @@ Performance features:
   perf in userspace:   disabled
 ```
 
-## Confirm Arm SPE is available to Perf
+## Confirm Arm SPE is available to perf
 
-Run the following command to confirm SPE is available to Perf: 
+Run the following command to confirm SPE is available to `perf`: 
 
 ```bash
 sudo perf list "arm_spe*"
@@ -99,32 +102,34 @@ List of pre-defined events (to be used in -e or -M):
   arm_spe_0//                                        [Kernel PMU event]
 ```
 
-Assign capabilities to Perf by running:
+Assign capabilities to `perf` by running:
 
 ```bash
 sudo setcap cap_perfmon,cap_sys_ptrace,cap_sys_admin+ep $(which perf)
 ```
 
-If `arm_spe` is not available because of your system configuration or if you don't have PMU permission, the `perf c2c` command will fail. 
+If `arm_spe` isn’t available due to your system configuration or limited PMU access, the `perf c2c` command will fail.
 
-To confirm Perf can access SPE run:
+To confirm `perf` can access SPE, run:
 
 ```bash
 perf c2c record
 ```
 
-The output showing the failure is:
+If SPE access is blocked, you’ll see output like this:
 
 ```output
 failed: memory events not supported
 ```
 
 {{% notice Note %}}
-If you are unable to use SPE it may be a restriction based on your cloud instance size or operating system.
+If you are unable to use SPE it might be a restriction based on your cloud instance size or operating system.
 
-Generally, access to a full server (also known as metal instances) with a relatively new kernel is needed for Arm SPE support. 
+Generally, access to a full server (also known as metal instances) with a relatively new kernel is required for Arm SPE support. 
 
 For more information about enabling SPE, see the [perf-arm-spe manual page](https://man7.org/linux/man-pages/man1/perf-arm-spe.1.html)
 {{% /notice %}}
 
-Continue to learn how to use Perf C2C on an example application.
+## Summary
+
+You've confirmed that your system supports Arm SPE, installed the necessary tools, and verified that `perf` can access SPE events. You're now ready to start collecting detailed performance data using `perf c2c`. In the next section, you’ll run a real application and use `perf c2c` to capture cache sharing behavior and uncover memory performance issues.

content/learning-paths/servers-and-cloud-computing/false-sharing-arm-spe/how-to-3.md

@@ -1,5 +1,5 @@
 ---
-title: False Sharing Example
+title: False sharing example
 weight: 4
 
 ### FIXED, DO NOT MODIFY
@@ -8,6 +8,10 @@ layout: learningpathall
 
 ## Example code
 
+{{% notice Learning Goal%}}
+The example code in this section demonstrates how false sharing affects performance by comparing two multithreaded programs; one with cache-aligned data structures, and one without. You’ll compile and run both versions, observe the runtime difference, and learn how memory layout affects cache behavior. This sets the stage for analyzing performance with `perf c2c` in the next section.
+{{% /notice %}}
+
 Use a text editor to copy and paste the C example code below into a file named `false_sharing_example.c`
 
 The code is adapted from [Joe Mario](https://github.com/joemario/perf-c2c-usage-files) and is discussed thoroughly in the Arm Statistical Profiling Extension Whitepaper.
@@ -285,7 +289,7 @@ int main ( int argc, char *argv[] )
 
 ### Code explanation
 
-The key data structure that occupies the cache is `struct Buf`. With a 64-byte cache line size, each line can hold 8, 8-byte `long` integers. 
+The key data structure that occupies the cache is `struct _buf`. With a 64-byte cache line size, each line can hold 8, 8-byte `long` integers. 
 
 If you do not pass in the `NO_FALSE_SHARING` macro during compilation the `Buf` data structure will contain the elements below. Each structure neatly occupies the entire 64-byte cache line. 
 
@@ -306,7 +310,7 @@ typedef struct _buf {
 
 Alternatively if you pass in the `NO_FALSE_SHARING` macro during compilation, the `Buf` structure has a different shape. 
 
-The 40 bytes of padding pushes the reader variables onto a different cache line. However, notice that this is with the tradeoff the new `Buf` structures occupies multiple cache lines (12 long integers). Therefore it leaves unused cache space of 25% per `Buf` structure.
+The 40 bytes of padding pushes the reader variables onto a different cache line. However, notice that this is with the tradeoff the new `Buf` structures occupies multiple cache lines (12 long integers). Therefore it leaves unused cache space of 25% per `Buf` structure. This trade-off uses more memory but eliminates false sharing, improving performance by reducing cache line contention.
 
 ```output
 typedef struct _buf {
@@ -345,5 +349,6 @@ user    0m8.869s
 sys     0m0.000s
 ```
 
-Continue to the next section to learn how to use Perf C2C to analyze the example code.
+## Summary
+In this section, you ran a hands-on C example to see how false sharing can significantly degrade performance in multithreaded applications. By comparing two versions of the same program, one with aligned memory access and one without, you saw how something as subtle as cache line layout can result in a 2x difference in runtime. This practical example sets the foundation for using Perf C2C to capture and analyze real cache line sharing behavior in the next section.