You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/bolt-merge/_index.md
-4Lines changed: 0 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,10 +1,6 @@
1
1
---
2
2
title: Optimize Arm applications and shared libraries with BOLT
3
3
4
-
draft: true
5
-
cascade:
6
-
draft: true
7
-
8
4
minutes_to_complete: 30
9
5
10
6
who_is_this_for: Performance engineers and software developers working on Arm platforms who want to optimize both application binaries and shared libraries using BOLT.
Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/bolt-merge/how-to-1.md
+11-4Lines changed: 11 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,11 +10,13 @@ layout: learningpathall
10
10
11
11
Make sure you have [BOLT](/install-guides/bolt/) and [Linux Perf](/install-guides/perf/) installed.
12
12
13
-
You should use an Arm Linux system with at least 4 CPUs and 16 Gb of RAM. Ubuntu 24.04 is used for testing, but other Linux distributions are possible.
13
+
You should use an Arm Linux system with at least 8 CPUs and 16 Gb of RAM. Ubuntu 24.04 is used for testing, but other Linux distributions are possible.
14
14
15
15
## What will I do in this Learning Path?
16
16
17
-
In this Learning Path you learn how to use BOLT to optimize applications and shared libraries. MySQL is used as the applcation and two share libraries which are used by MySQL are also optimized using BOLT.
17
+
In this Learning Path you learn how to use BOLT to optimize applications and shared libraries. MySQL is used as the application and two share libraries which are used by MySQL are also optimized using BOLT.
18
+
19
+
Here is an outline of the steps:
18
20
19
21
1. Collect and merge BOLT profiles from multiple workloads, such as read-only and write-only
20
22
@@ -36,18 +38,23 @@ In this Learning Path you learn how to use BOLT to optimize applications and sha
36
38
37
39
After optimizing each component, you combine them to create a deployment where both the application and its libraries benefit from BOLT's enhancements.
38
40
41
+
## What is BOLT profile merging?
42
+
43
+
BOLT profile merging is the process of combining profiling from multiple runs into a single profile. This merged profile enables BOLT to optimize binaries for a broader set of real-world behaviors, ensuring that the final optimized application or library performs well across diverse workloads, not just a single use case. By merging profiles, you capture a wider range of code paths and execution patterns, leading to more robust and effective optimizations.
44
+
45
+
39
46
40
47
## What are good applications for BOLT?
41
48
42
-
MySQL and sysbench are used as example applications, but you can use this method for **any feature-rich application** that:
49
+
MySQL and Sysbench are used as example applications, but you can use this method for any feature-rich application that:
43
50
44
51
1. Exhibits multiple runtime paths
45
52
46
53
Applications often have different code paths depending on the workload or user actions. Optimizing for just one path can leave performance gains untapped in others. By profiling and merging data from various workloads, you ensure broader optimization coverage.
47
54
48
55
2. Uses dynamic libraries
49
56
50
-
Many modern applications rely on shared libraries for functionality. Optimizing these libraries alongside the main binary ensures consistent performance improvements throughout the application.
57
+
Most modern applications rely on shared libraries for functionality. Optimizing these libraries alongside the main binary ensures consistent performance improvements throughout the application.
51
58
52
59
3. Requires full-stack binary optimization for performance-critical deployment
After the build completes, the `mysqld` binary is located at `$HOME/mysql-server/build/runtime_output_directory/mysqld`
49
+
50
+
{{% notice Note %}}
51
+
You can run `mysqld` directly from the build directory as shown, or run `make install` to install it system-wide. For testing and instrumentation, running from the build directory is usually preferred.
52
+
{{% /notice %}}
53
+
54
+
After building mysqld, install MySQL server and client utilities system-wide:
55
+
56
+
```bash
57
+
sudo make install
58
+
```
59
+
60
+
This will make the `mysql` client and other utilities available in your PATH.
61
+
62
+
Ensure the binary is unstripped and includes debug symbols for BOLT instrumentation.
63
+
64
+
To work with BOLT, your application binary should be:
65
+
66
+
- Built from source
18
67
- Unstripped, with symbol information available
19
68
- Compiled with frame pointers enabled (`-fno-omit-frame-pointer`)
20
69
21
70
You can verify this with:
22
71
23
72
```bash
24
-
readelf -s /path/to/mysqld | grep main
73
+
readelf -s $HOME/mysql-server/build/runtime_output_directory/mysqld | grep main
74
+
```
75
+
76
+
The partial output is:
77
+
78
+
```output
79
+
23837: 000000000950dfe8 8 OBJECT GLOBAL DEFAULT 27 mysql_main
80
+
34522: 000000000915bfd0 8 OBJECT GLOBAL DEFAULT 26 server_main_callback
81
+
42773: 00000000051730e4 80 FUNC GLOBAL DEFAULT 13 _Z18my_main_thre[...]
82
+
44882: 000000000357dc98 40 FUNC GLOBAL DEFAULT 13 main
83
+
61046: 0000000005ffd5c0 40 FUNC GLOBAL DEFAULT 13 _Z21record_main_[...]
25
84
```
26
85
27
86
If the symbols are missing, rebuild the binary with debug info and no stripping.
28
87
29
-
### Step 2: Instrument the binary with BOLT
88
+
## Instrument the binary with BOLT
30
89
31
90
Use `llvm-bolt` to create an instrumented version of the binary:
-`--instrumentation-file`: Path where the profile output will be saved
47
106
-`--instrumentation-wait-forks`: Ensures the instrumentation continues through forks (important for daemon processes)
48
107
49
-
---
50
108
51
-
### Step 3: Run the instrumented binary under a feature-specific workload
109
+
## Start the instrumented MySQL server
110
+
111
+
Before running the workload, start the instrumented MySQL server in a separate terminal. You may need to initialize a new data directory if this is your first run:
Adjust `--datadir`, `--socket`, and `--port` as needed for your environment. Make sure the server is running and accessible before proceeding.
127
+
128
+
## Install sysbench
129
+
130
+
You will need sysbench to generate workloads for MySQL. On most Arm Linux distributions, you can install it using your package manager:
131
+
132
+
```bash
133
+
sudo apt update
134
+
sudo apt install -y sysbench
135
+
```
136
+
137
+
Alternatively, see the [sysbench GitHub page](https://github.com/akopytov/sysbench) for build-from-source instructions if a package is not available for your platform.
138
+
139
+
## Create a test database and user
140
+
141
+
For sysbench to work, you need a test database and user. Connect to the MySQL server as the root user (or another admin user) and run:
142
+
143
+
```bash
144
+
mysql -u root --socket=$HOME/mysql-bolt.sock
145
+
```
146
+
147
+
Then, in the MySQL shell:
148
+
149
+
```sql
150
+
CREATEDATABASEIF NOT EXISTS bench;
151
+
CREATEUSERIF NOT EXISTS 'bench'@'localhost' IDENTIFIED BY 'bench';
152
+
GRANT ALL PRIVILEGES ON bench.* TO 'bench'@'localhost';
153
+
FLUSH PRIVILEGES;
154
+
EXIT;
155
+
```
156
+
157
+
## Run the instrumented binary under a feature-specific workload
52
158
53
159
Use a workload generator to stress the binary in a feature-specific way. For example, to simulate **read-only traffic** with sysbench:
54
160
55
161
```bash
56
-
taskset -c 9 ./src/sysbench \\
57
-
--db-driver=mysql \\
58
-
--mysql-host=127.0.0.1 \\
59
-
--mysql-db=bench \\
60
-
--mysql-user=bench \\
61
-
--mysql-password=bench \\
62
-
--mysql-port=3306 \\
63
-
--tables=8 \\
64
-
--table-size=10000 \\
65
-
--threads=1 \\
66
-
src/lua/oltp_read_only.lua run
162
+
taskset -c 7 sysbench \
163
+
--db-driver=mysql \
164
+
--mysql-host=127.0.0.1 \
165
+
--mysql-db=bench \
166
+
--mysql-user=bench \
167
+
--mysql-password=bench \
168
+
--mysql-port=3306 \
169
+
--tables=8 \
170
+
--table-size=10000 \
171
+
--threads=1 \
172
+
/usr/share/sysbench/oltp_read_only.lua run
67
173
```
68
174
69
-
> Adjust this command as needed for your workload and CPU/core binding.
175
+
{{% notice Note %}}
176
+
On an 8-core system, cores are numbered 0-7. Adjust the `taskset -c` values as needed for your system. Avoid using the same core for both mysqld and sysbench to reduce contention.
177
+
{{% /notice %}}
70
178
71
-
The `.fdata` file defined in `--instrumentation-file` will be populated with runtime execution data.
72
179
73
-
---
180
+
The `.fdata` file defined in `--instrumentation-file` will be populated with runtime execution data.
74
181
75
-
### Step 4: Verify the profile was created
182
+
## Verify the profile was created
76
183
77
184
After running the workload:
78
185
79
186
```bash
80
-
ls -lh /path/to/profile-readonly.fdata
187
+
ls -lh $HOME/mysql-server/build/profile-readonly.fdata
81
188
```
82
189
83
190
You should see a non-empty file. This file will later be merged with other profiles (e.g., for write-only traffic) to generate a complete merged profile.
Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/bolt-merge/how-to-3.md
+25-29Lines changed: 25 additions & 29 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,54 +1,53 @@
1
1
---
2
-
title: BOLT Optimization - Second Feature & BOLT Merge to combine
2
+
title: Run a new workload using BOLT and merge the results
3
3
weight: 4
4
4
5
5
### FIXED, DO NOT MODIFY
6
6
layout: learningpathall
7
7
---
8
8
9
-
In this step, you'll collect profile data for a **write-heavy** workload and also **instrument external libraries** such as `libcrypto.so` and `libssl.so` used by the application (e.g., MySQL).
9
+
Next, you will collect profile data for a **write-heavy** workload and merge the results with the **read-heavy** workload in the previous section.
10
10
11
-
12
-
### Step 1: Run Write-Only Workload for Application Binary
11
+
## Run Write-Only Workload for Application Binary
13
12
14
13
Use the same BOLT-instrumented MySQL binary and drive it with a write-only workload to capture `profile-writeonly.fdata`:
15
14
16
15
```bash
17
-
taskset -c 9 ./src/sysbench \\
18
-
--db-driver=mysql \\
19
-
--mysql-host=127.0.0.1 \\
20
-
--mysql-db=bench \\
21
-
--mysql-user=bench \\
22
-
--mysql-password=bench \\
23
-
--mysql-port=3306 \\
24
-
--tables=8 \\
25
-
--table-size=10000 \\
26
-
--threads=1 \\
27
-
src/lua/oltp_write_only.lua run
16
+
# On an 8-core system, use available cores (e.g., 7 for sysbench)
17
+
taskset -c 7 sysbench \
18
+
--db-driver=mysql \
19
+
--mysql-host=127.0.0.1 \
20
+
--mysql-db=bench \
21
+
--mysql-user=bench \
22
+
--mysql-password=bench \
23
+
--mysql-port=3306 \
24
+
--tables=8 \
25
+
--table-size=10000 \
26
+
--threads=1 \
27
+
/usr/share/sysbench/oltp_write_only.lua run
28
28
```
29
29
30
30
Make sure that the `--instrumentation-file` is set appropriately to save `profile-writeonly.fdata`.
31
-
---
32
-
### Step 2: Verify the Second Profile Was Generated
31
+
32
+
33
+
### Verify the Second Profile Was Generated
33
34
34
35
```bash
35
-
ls -lh /path/to/profile-writeonly.fdata
36
+
ls -lh $HOME/mysql-server/build/profile-writeonly.fdata
36
37
```
37
38
38
39
Both `.fdata` files should now exist and contain valid data:
39
40
40
41
-`profile-readonly.fdata`
41
42
-`profile-writeonly.fdata`
42
43
43
-
---
44
-
45
-
### Step 3: Merge the Feature Profiles
44
+
### Merge the Feature Profiles
46
45
47
46
Use `merge-fdata` to combine the feature-specific profiles into one comprehensive `.fdata` file:
0 commit comments