Skip to content

Commit 48035d2

Browse files
author
Your Name
committed
final review
1 parent 71427a6 commit 48035d2

File tree

4 files changed

+22
-18
lines changed

4 files changed

+22
-18
lines changed

content/learning-paths/cross-platform/filesystem-cache-hit/_index.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
---
2-
title: Improve File System Cache hit rate with the posix_fadvise syscall
2+
title: Improve File System Cache hit rate with the posix_fadvise function
33

44
minutes_to_complete: 15
55

6-
who_is_this_for: Developers who observe high-levels of file system cache misses in performance-critical sections of software
6+
who_is_this_for: Developers who want to boost performance of applications limited by file system cache misses.
77

88
learning_objectives:
99
- Learn the high-level operation and purpose of the file system cache
1010
- Learn how to measure and intepret cache miss rates
11-
- Learn how to use the posix_fadvise syscall provide hints to the kernel about file access patterns
11+
- Learn how to use the posix_fadvise() function to provide hints to the kernel about file access patterns
1212

1313
prerequisites:
1414
- Basic understanding of C++ and Linux

content/learning-paths/cross-platform/filesystem-cache-hit/intro.md

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -8,31 +8,32 @@ layout: learningpathall
88

99
## Recap of File Systems
1010

11-
A file system is the method an operating system uses to store, organize, and manage data on storage devices like hard drives or SSDs. Linux uses a **Virtual File System (VFS)** layer to provide a uniform interface to different file system types (e.g., `ext4`, `xfs`, `btrfs`), allowing programs to interact with them in a consistent way.
11+
A file system is the method an operating system uses to store, organize, and manage data on storage devices like hard drives or SSDs. Linux uses a virtual file system (VFS) layer to provide a uniform interface to different file system types (e.g., `ext4`, `xfs`, `btrfs`), allowing programs to interact with them in a consistent way.
1212

13-
Developers typically interact with the file system through system calls or standard library functions—for example, using `open()`, `read()`, and `write()` in C to access files. For instance, reading a configuration file at `/etc/myapp/config.json` involves navigating the file system hierarchy and accessing the file’s contents through these interfaces. To speed up access to such files, the operating system creates a file system cache, managed by the kernel and residing in main memory (RAM). This cache temporarily stores recently accessed file data to speed up future reads and reduce disk I/O.
13+
Developers typically interact with the file system through system calls or standard library functions—for example, using `open()`, `read()`, and `write()` in C to access files. For instance, reading a configuration file at `/etc/myapp/config.json` involves navigating the file system hierarchy and accessing the file’s contents through these interfaces. To speed up access to such files, the operating system creates a file system cache, managed by the kernel and resides in main memory (RAM). This cache temporarily stores recently accessed file data and metadata to speed up future reads and reduce disk I/O.
1414

1515

16-
## Observing Memory Usage
16+
## Recap of Memory Usage
1717

18-
The operating system has to manage the memory usage for the entire system.
18+
The hardware and operating system is responsible for managing the memory usage of the system.
1919

20-
The command `free -h` provides a snapshot of memory usage.
20+
The well-known command `free -wh` provides a snapshot of memory usage. The `cache` column includes the portion of memory used to store the file system cache.
2121

2222
```output
23-
total used free shared buff/cache available
24-
Mem: 7.6Gi 533Mi 7.0Gi 948Ki 215Mi 7.1Gi
23+
total used free shared buffers cache available
24+
Mem: 7.6Gi 1.1Gi 5.8Gi 960Ki 22Mi 832Mi 6.5Gi
2525
Swap: 0B 0B 0B
2626
```
2727

28-
The `free` command summarises the memory usage into the following columns.
28+
The command summarises the memory usage into the following columns.
2929

3030
- `total`: Total installed memory
3131
- `used`: Memory in use (excluding cache/buffers)
3232
- `free`: Unused memory
3333
- `shared`: Memory used by tmpfs and shared between processes
34-
- `buff/cache`: Sum of memory used by kernel buffers and cache
35-
- `available`: An estimate of memory available to start new apps
34+
- `buffers`: Memory used by kernel buffers
35+
- `cache`: Memory used by file system cache
36+
- `available`: An estimate of memory available (both free memory and memory that can be reclaimed) when a new process starts.
3637

3738

3839
### What is the posix_fadvise syscall?

content/learning-paths/cross-platform/filesystem-cache-hit/with_hint.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ layout: learningpathall
99

1010
## Providing hints with posix_fadvise
1111

12-
The `posix_fadvise()` system call gives the Linux kernel hints about expected file access patterns to optimize I/O. The function takes the following arguments.
12+
The `posix_fadvise()` function is a wrapper around the `fadvise64` linux system call (syscall). This syscall gives the Linux kernel hints about expected file access patterns to optimize I/O, for example preemptively reading ahead for sequential file access. The function takes the following arguments.
1313
- `fd`: file descriptor of the file,
1414
- `offset`: where the advice starts (0 = beginning),
1515
- `len`: how many bytes the advice applies to (0 = to the end),
@@ -58,7 +58,6 @@ int main() {
5858
return 0;
5959
}
6060

61-
6261
```
6362

6463
Compile with the following command.
@@ -83,10 +82,14 @@ sudo perf stat -e cache-references,cache-misses,minor-faults,major-faults -r 9 .
8382
8483
```
8584

85+
{{% notice Tip%}}
86+
If you are using `posix_fadvise()` with your own application and you want to observe which system calls are issued. Consider using the system call tracer, `strace` with a command such as `strace -ttT -e trace=<syscall of interest,fadvise64> ./<your workload>` to observe what is causing fewer cache misses.
87+
{{% /notice %}}
88+
8689
### Results
8790

8891
Here we observe that on this run with a single line of code we are able to reduce the cache miss rate from ~4.8% to ~3.8%. This can translate to more efficient and performant software, especially if your program is synchronous and has to wait for disk accesses.
8992

9093
{{% notice Please Note%}}
91-
Since this is only a hint to the operating system and it depends on other factors such as memory pressure, memory usage, other processes etc. the behaviour on your system may be different
94+
Since this is advise to the operating system, the operating may not do anything with this. Real-world impact depends on other factors such as memory pressure, memory usage etc. As such, the behaviour on your own system may be different.
9295
{{% /notice %}}

content/learning-paths/cross-platform/filesystem-cache-hit/without_hint.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ layout: learningpathall
88

99
## Setup
1010

11-
For this demonstration, I have connected to an Arm-based AWS `c7g.xlarge` instance running Ubuntu 24.04. Results may vary depending on which instance and kernel version you are using. At the time of writing, I was using the `6.8.0-1024-aws` kernel version.
11+
For this demonstration, connect to an Arm-based AWS `c7g.xlarge` instance running Ubuntu 24.04. Results may vary depending on which instance and kernel version you are using. At the time of writing, kernel version `6.8.0-1024-aws` was used.
1212

1313
First, you need to install the linux performance measure tool, `perf`. Please follow the [installation guide](https://learn.arm.com/install-guides/perf/) for your system.
1414

@@ -33,7 +33,7 @@ L2 cache: 4 MiB (4 instances)
3333
L3 cache: 32 MiB (1 instance)
3434
```
3535

36-
This information will be useful as we want to ensure our working set size cannot all fit within on-CPU cache.
36+
This information will be useful to ensure our working set size cannot all fit within on-CPU cache.
3737

3838
Next, check the current memory usage of an idle system with the `free -h` command.
3939

0 commit comments

Comments
 (0)