Skip to content

Commit 7e271e3

Browse files
committed
Tech review part sweet part1
1 parent 1c0de35 commit 7e271e3

File tree

7 files changed

+58
-80
lines changed

7 files changed

+58
-80
lines changed

content/learning-paths/servers-and-cloud-computing/go-benchmarking-with-sweet/_index.md

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,21 @@
11
---
22
title: Go Benchmarks with Sweet and Benchstat
33

4+
draft: true
5+
cascade:
6+
draft: true
7+
48
minutes_to_complete: 60
59

6-
who_is_this_for: This learning path is for developers who are interested in measuring the performance of Go-based applications on their servers.
10+
who_is_this_for: This is an introductory topic for developers who are interested in measuring the performance of Go-based applications on Arm-based servers.
711

812
learning_objectives:
9-
- Bring up two instances of GCP VMs
13+
- Learn how to start up Arm64 and x64 instances of GCP VMs
1014
- Install Go, benchmarks, benchstat, and sweet on the two VMs
11-
- Using sweet and benchstat, compare performance of Go applications on the two VMs
15+
- Use sweet and benchstat to compare the performance of Go applications on the two VMs
1216

1317
prerequisites:
14-
- A [Google Cloud account](https://console.cloud.google.com/). This learning path can be run on-prem or on any cloud provider, but specifically documents the process on GCP's Google Axion platform.
18+
- A [Google Cloud account](https://console.cloud.google.com/). This learning path can be run on on-prem or on any cloud provider instance, but specifically documents the process for running on Google Axion.
1519
- A local machine with [Google Cloud CLI](/install-guides/gcloud/) installed.
1620

1721
author: Geremy Cohen
@@ -21,8 +25,9 @@ skilllevels: Introductory
2125
subjects: Performance and Architecture
2226
armips:
2327
- Neoverse
28+
cloud_service_providers: Google Cloud
2429
tools_software_languages:
25-
30+
- Go
2631
operatingsystems:
2732
- Linux
2833

content/learning-paths/servers-and-cloud-computing/go-benchmarking-with-sweet/add_c4_vm.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
---
2-
title: Launching an Intel C4 Instance
2+
title: Launching a Intel Emerald Rapids Instance
33
weight: 30
44

55
### FIXED, DO NOT MODIFY
66
layout: learningpathall
77
---
88

99
## Section Overview
10-
In this section, you will set up the second benchmarking system, an Intel-based c4-standard-8 instance.
10+
In this section, you will set up the second benchmarking system, an Intel Emerald Rapids `c4-standard-8` instance.
1111

1212
## Creating the Instance
1313

@@ -27,4 +27,4 @@ To create the second system, follow the previous lesson's c4a install instructio
2727

2828
![](images/launch_c4/5.png)
2929

30-
After the c4 instance starts up, you are ready to continue to the next section, where you'll install the benchmarking software.
30+
After the c4 instance starts up, you are ready to continue to the next section, where you'll install the benchmarking software.

content/learning-paths/servers-and-cloud-computing/go-benchmarking-with-sweet/add_c4a_vm.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
---
2-
title: Launching an Arm Axion C4a Instance
2+
title: Launching a Google Axion Instance
33
weight: 20
44

55
### FIXED, DO NOT MODIFY
66
layout: learningpathall
77
---
88

9-
## Section Overview
10-
In this section, you'll learn how to spin up the first of two different systems used in our benchmarking tests, an Arm-based Google Axion c4a-standard-4 (c4a for short).
9+
## Overview
10+
In this section, you'll learn how to spin up the first of two different VMs used for benchmarking Go tests, an Arm-based Google Axion c4a-standard-4 (c4a for short).
1111

1212
## Creating the c4a-standard-4 Instance
1313

@@ -63,5 +63,5 @@ The Machine configuration page appears.
6363

6464
![](images/launch_c4a/19.png)
6565

66-
After a few seconds, your c4a instance starts up, and you are ready to continue to the next section. In the next step, you will launch the second system, an Intel-based Emerald Rapids c4-standard-8 (c4 for short), which will serve as the comparison system for our benchmarking tests.
66+
After a few seconds, your c4a instance starts up, and you are ready to continue to the next section. In the next step, you will launch the second VM, an Intel-based Emerald Rapids c4-standard-8 (c4 for short), which will serve as the comparison system for our benchmarking tests.
6767

content/learning-paths/servers-and-cloud-computing/go-benchmarking-with-sweet/installing_go_and_sweet.md

Lines changed: 27 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -6,21 +6,19 @@ weight: 40
66
layout: learningpathall
77
---
88

9-
## Section Overview
10-
119
In this section, you'll install Go, Sweet, and the Benchstat comparison tool on both VMs.
1210

1311
## Installation Script
1412

15-
Sweet is a Go benchmarking tool that provides a standardized way to run performance tests across different systems. Benchstat is a companion tool that analyzes and compares benchmark results, helping you understand performance differences between systems. Together, these tools will allow us to accurately measure and compare Go performance on Arm and x86 architectures.
13+
Sweet is a Go benchmarking tool that provides a standardized way to run performance tests across different systems. Benchstat is a companion tool that analyzes and compares benchmark results, helping you understand performance differences between systems. Together, these tools will allow you to accurately measure and compare Go performance on Arm and x86 architectures.
1614

1715

1816
{{% notice Note %}}
19-
Subsequent steps in the learning path assume you are running this script (installing) from your home directory (`~`), resulting in the creation of a `~/benchmarks/sweet` final install path. If you decide to install elsewhere, adjust the path accordingly when prompted to run the benchmark logic later in the learning path.
17+
Subsequent steps in the learning path assume you are running this script (installing) from your home directory (`$HOME`), resulting in the creation of a `$HOME/benchmarks/sweet` final install path. If you decide to install elsewhere, you will need to adjust the path accordingly when prompted to run the benchmark logic later in the learning path.
2018
{{% /notice %}}
2119

2220

23-
Copy and paste this script to **both** of your GCP VMs.
21+
Start by copying and pasting the script below on **both** of your GCP VMs. This script checks the architecture of your running VM, installs the required Go packge on your VM. It then installs sweet, benchmarks, and the benchstat tools.
2422

2523
**You don't need to run it after pasting**, just paste it into your home directory and press enter to install all needed dependencies:
2624

@@ -92,18 +90,39 @@ chmod 755 install_go_and_sweet.sh
9290

9391
```
9492

93+
The end of the output should look like:
94+
95+
```output
96+
Sweet v0.3.0: Go Benchmarking Suite
97+
98+
Retrieves assets for benchmarks from GCS.
99+
100+
Usage: sweet get [flags]
101+
-cache string
102+
cache location for assets (default "/home/pareena_verma_arm_com/.cache/go-sweet")
103+
-clean
104+
delete all cached assets before installing new ones
105+
-copy string
106+
location to extract assets into, useful for development
107+
-version string
108+
the version to download assets for (default "v0.3.0")
109+
```
95110

96111

97112
## Verify Installation
98113

99-
To test that everything is installed correctly, run the following command on each VM after the script completes:
114+
To test that everything is installed correctly, set the environment variables shown below on each VM:
100115

101116
```bash
102-
# Run the benchmarks
103117
export GOPATH=$HOME/go
104118
export GOBIN=$GOPATH/bin
105119
export PATH=$PATH:$GOBIN:/usr/local/go/bin
120+
```
121+
Now run the `markdown` benchmark with `sweet` on both VMs as shown:
122+
123+
```bash
106124
cd benchmarks/sweet
125+
sweet get
107126
sweet run -count 10 -run="markdown" config.toml # run one, 1X
108127
```
109128

@@ -117,4 +136,4 @@ You should see output similar to the following:
117136
[sweet] Running benchmark markdown for arm-benchmarks: run 1
118137
...
119138
[sweet] Running benchmark markdown for arm-benchmarks: run 10
120-
```
139+
```

content/learning-paths/servers-and-cloud-computing/go-benchmarking-with-sweet/manual_run_benchmark.md

Lines changed: 9 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -6,59 +6,29 @@ weight: 51
66
layout: learningpathall
77
---
88

9-
## Section Overview
9+
In this section, you'll download the results of the benchmark you ran manually in the previous sections from each VM. You will use these results to understand how `sweet` and `benchstat` work together.
1010

11-
In this section, you'll run benchmarks manually to understand how `sweet` and `benchstat` work together.
11+
## Download Benchmark Results from each VM
12+
Lets walk through the steps to manually download the sweet benchmark results from your initial run on each VM.
1213

13-
## Run Benchmarks on Each Machine
14-
The following explains how to run the benchmarks by hand on each machine.
1514

16-
1. **Access VM instances:** Navigate to the GCP [VM Instances](https://console.cloud.google.com/compute/instances) console.
17-
18-
2. **Connect to c4a instance:** Click on the `SSH` button next to your `c4a` instance.
19-
An SSH terminal will open in a new tab.
20-
21-
![](images/run_manually/2.png)
22-
23-
3. **Set up environment:** Copy and paste the following into the SSH terminal to setup the environment and change to the sweet directory:
24-
25-
```bash
26-
cd benchmarks/sweet
27-
export GOPATH=$HOME/go
28-
export GOBIN=$GOPATH/bin
29-
export PATH=$PATH:$GOBIN:/usr/local/go/bin
30-
```
31-
32-
![](images/run_manually/3.png)
33-
34-
{{% notice Note %}}
35-
The above instructions assume you installed the benchmarks in the `~/benchmarks/sweet` directory. If you installed them elsewhere, adjust the path accordingly.
36-
{{% /notice %}}
37-
38-
39-
4. **Run the benchmark:** Copy and paste the following command to run the `markdown` benchmark with `sweet`:
40-
41-
```bash
42-
sweet run -count 10 -run="markdown" config.toml
43-
```
44-
45-
5. **Locate results:** After the benchmark completes, cd to the `results/markdown` directory and list the files to see the `arm-benchmarks.result` file:
15+
1. **Locate results:** Change directory to the `results/markdown` directory and list the files to see the `arm-benchmarks.result` file:
4616

4717
```bash
4818
cd results/markdown
4919
ls -d $PWD/*
5020
```
5121

52-
6. **Copy result path:** Copy the absolute pathname of `arm-benchmarks.result`.
22+
2. **Copy result path:** Copy the absolute pathname of `arm-benchmarks.result`.
5323

54-
7. **Download results:** Click `DOWNLOAD FILE`, and paste the **ABSOLUTE PATHNAME** you just copied for the filename, and then click `Download`. This will download the benchmark results to your local machine.
24+
3. **Download results:** Click `DOWNLOAD FILE`, and paste the **ABSOLUTE PATHNAME** you just copied for the filename, and then click `Download`. This will download the benchmark results to your local machine.
5525

5626
![](images/run_manually/6.png)
5727

58-
8. **Rename the file:** Once downloaded, on your local machine, rename this file to `c4a.result` so you can distinguish it from the x86 results you'll download later. This naming convention will help you clearly identify which results came from which architecture. You'll know the file downloaded successfully if you see the file in your Downloads directory with the name `c4a.result`, as well as the confirmation dialog in your browser:
28+
4. **Rename the file:** Once downloaded, on your local machine, rename this file to `c4a.result` so you can distinguish it from the x86 results you'll download later. This naming convention will help you clearly identify which results came from which architecture. You'll know the file downloaded successfully if you see the file in your Downloads directory with the name `c4a.result`, as well as the confirmation dialog in your browser:
5929

6030
![](images/run_manually/7.png)
6131

62-
9. **Repeat for c4 instance:** Repeat steps 2-8 with your `c4` (x86) instance. Do everything the same, except after downloading the c4's `arm-benchmarks.result` file, rename it to `c4.result`.
32+
5. **Repeat for c4 instance:** Repeat steps 2-8 with your `c4` (x86) instance. Do everything the same, except after downloading the c4's `arm-benchmarks.result` file, rename it to `c4.result`.
6333

64-
Now that you have the results from both VMs, in the next section, you'll learn how to use benchstat to analyze these results and understand the performance differences between the two architectures.
34+
Now that you have the results from both VMs, in the next section, you'll learn how to use benchstat to analyze these results and understand the performance differences between the two architectures.

content/learning-paths/servers-and-cloud-computing/go-benchmarking-with-sweet/overview.md

Lines changed: 2 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,9 @@ weight: 10
66
layout: learningpathall
77
---
88

9-
# Go Benchmarking Learning Path Overview
9+
# Go Benchmarking Overview
1010

11-
In this module, you'll learn how to measure, collect, and compare Go performance data across different CPU architectures. This knowledge is essential for developers and system architects who need to make informed decisions about infrastructure choices for their Go applications.
11+
In this section, you will learn how to measure, collect, and compare Go performance data across different CPU architectures. This knowledge is essential for developers and system architects who need to make informed decisions about infrastructure choices for their Go applications.
1212

1313
You'll gain hands-on experience with:
1414

@@ -31,18 +31,4 @@ Arm-based c4a-standard-4 instances and Intel-based c4-standard-8 instances both
3131
{{% /notice %}}
3232

3333

34-
## Prerequisites
3534

36-
Before starting this learning path, you should have:
37-
- A Google Cloud Platform account with permissions to create VMs
38-
- Basic familiarity with Linux command line
39-
- Basic understanding of what benchmarking is and why it's useful
40-
41-
## What You'll Accomplish
42-
43-
When you are finished running through this learning path, you'll have:
44-
45-
1. Brought up Arm and x86 instances of GCP-based VMs
46-
2. Installed Go, benchmarks, benchstat, and sweet on the two VMs
47-
3. Used sweet and benchstat to compare performance of Go applications on the two VMs
48-
4. Extrapolated the knowledge to create Go Benchmarks for your own workloads and systems

content/learning-paths/servers-and-cloud-computing/go-benchmarking-with-sweet/running_benchmarks.md

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,11 @@ weight: 50
66
layout: learningpathall
77
---
88

9-
## Section Overview
10-
11-
With setup complete, you can now run and analyze the benchmarks. Before you do, it's good to understand all the moving pieces in more detail.
9+
With setup complete, you can now run and analyze the benchmarks. Before you do, it's good to understand all the different pieces in more detail.
1210

1311
## Choosing a Benchmark to Run
1412

15-
Whether running manually or automatic, the benchmarking process consists of two main steps:
13+
Whether running manually or automatically, the benchmarking process consists of two main steps:
1614

1715
1. **Running benchmarks with Sweet**: `sweet` executes the benchmarks on each VM, generating raw performance data
1816

@@ -99,7 +97,7 @@ Large gaps between average and peak memory usage suggest opportunities for memor
9997

10098
Here are some general tips to keep in mind as you explore benchmarking across different apps and instance types:
10199

102-
- Unlike Intel and AMD processors that use hyper-threading, Arm processors provide single-threaded cores without hyper-threading. A four-core Arm processor has four independent cores running four threads, while an four-core Intel processor provides eight logical cores through hyper-threading. This means each Arm vCPU represents a full physical core, while each Intel/AMD vCPU represents half a physical core. For fair comparison, this learning path uses a 4-vCPU Arm instance against an 8-vCPU Intel instance. When scaling up instance sizes during benchmarking, make sure to keep a 2:1 Intel/AMD:Arm VCPU ratio if you wish to keep parity on CPU resources.
100+
- Unlike Intel and AMD processors that use hyper-threading, Arm processors provide single-threaded cores without hyper-threading. A four-core Arm processor has four independent cores running four threads, while an four-core Intel processor provides eight logical cores through hyper-threading. This means each Arm vCPU represents a full physical core, while each Intel/AMD vCPU represents half a physical core. For fair comparison, this learning path uses a 4-vCPU Arm instance against an 8-vCPU Intel instance. When scaling up instance sizes during benchmarking, make sure to keep a 2:1 Intel/AMD:Arm vCPU ratio if you wish to keep parity on CPU resources.
103101

104102
- It's suggested to run each benchmark at least 10 times (specified via the `count` parameter) to handle outlier/errant runs and ensure statistical significance.
105103

0 commit comments

Comments
 (0)