Skip to content

Commit 76d3e64

Browse files
Stacyrch140Stacyrch140
authored
Merge pull request #231771 from MSFTeegarden/patch-18
(17056587) Update performance benchmark numbers
2 parents 7225e76 + d82ed31 commit 76d3e64

File tree

2 files changed

+146
-46
lines changed

2 files changed

+146
-46
lines changed

‎articles/azure-cache-for-redis/cache-best-practices-performance.md

Lines changed: 145 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -5,30 +5,49 @@ description: Learn how to test the performance of Azure Cache for Redis.
55
author: flang-msft
66
ms.service: cache
77
ms.topic: conceptual
8-
ms.date: 08/25/2021
8+
ms.date: 04/06/2022
99
ms.author: franlanglois
1010
---
1111

1212
# Performance testing
1313

14-
1. Start by using `redis-benchmark.exe` to check the general throughput and latency characteristics of your cache before writing your own performance tests. For more information, see [Redis-Benchmark](#redis-benchmark-utility).
14+
Testing the performance of a Redis instance can be a complicated task. The performance of a Redis instance can vary based on parameters such as the number of clients, the size of data values, and whether pipelining is being used. There also can be a tradeoff between optimizing throughput or latency.
1515

16-
1. The client VM used for testing should be *in the same region* as your Redis cache instance.
16+
Fortunately, several tools exist to make benchmarking Redis easier. Two of the most popular tools are **[redis-benchmark](https://redis.io/docs/management/optimization/benchmarks/)** and **[memtier-benchmark](https://github.com/redislabs/memtier_benchmark)**. This article focuses on redis-benchmark.
1717

18-
1. Make sure the client VM you use has *at least as much compute and bandwidth* as the cache being tested.
18+
## How to use the redis-benchmark utility
1919

20-
1. It's important that you don't test the performance of your cache only under steady state conditions. *Test under failover conditions too*, and measure the CPU/Server Load on your cache during that time. You can start a failover by [rebooting the primary node](cache-administration.md#reboot). Testing under failover conditions allows you to see the throughput and latency of your application during failover conditions. Failover can happen during updates or during an unplanned event. Ideally you don't want to see CPU/Server Load peak to more than say 80% even during a failover as that can affect performance.
20+
1. Install open source Redis server to a client VM you can use for testing. The redis-benchmark utility is built into the open source Redis distribution. Follow the [Redis documentation](https://redis.io/docs/getting-started/#install-redis) for instructions on how to install the open source image.
2121

22-
1. Consider using Premium tier Azure Cache for Redis instances. These cache sizes have better network latency and throughput because they're running on better hardware for both CPU and network.
22+
1. The client VM used for testing should be *in the same region* as your Azure Cache for Redis instance.
2323

24-
> [!NOTE]
25-
> Our observed performance results are [published here](./cache-planning-faq.yml#azure-cache-for-redis-performance) for your reference. Also, be aware that SSL/TLS adds some overhead, so you may get different latencies and/or throughput if you're using transport encryption.
24+
1. Make sure the client VM you use has *at least as much compute and bandwidth* as the cache instance being tested.
2625

27-
## Redis-benchmark utility
26+
1. Configure your [network isolation](cache-network-isolation.md) and [firewall](cache-configure.md#firewall) settings to ensure that the client VM is able to access your Azure Cache for Redis instance.
2827

29-
**Redis-benchmark** documentation can be [found here](https://redis.io/docs/reference/optimization/benchmarks/).
28+
1. If you're using TLS/SSL on your cache instance, you need to add the `--tls` parameter to your redis-benchmark command or use a proxy like [stunnel](https://www.stunnel.org/index.html).
3029

31-
The `redis-benchmark.exe` doesn't support TLS. You'll have to [enable the Non-TLS port through the Portal](cache-configure.md#access-ports) before you run the test. A Windows-compatible version of redis-benchmark.exe can be found [here](https://github.com/MSOpenTech/redis/releases).
30+
1. `Redis-benchmark` uses port 6379 by default. Use the `-p` parameter to override this setting. You need to do use `-p`, if you're using the SSL/TLS (port 6380) or are using the Enterprise tier (port 10000).
31+
32+
1. If you're using an Azure Cache for Redis instance that uses [clustering](cache-how-to-scale.md), you need to add the `--cluster` parameter to your `redis-benchmark` command. Enterprise tier caches using the [Enterprise clustering policy](cache-best-practices-enterprise-tiers.md#clustering-on-enterprise) can be treated as nonclustered caches and don't need this setting.
33+
34+
1. Launch `redis-benchmark` from the CLI or shell of the VM. For instructions on how to configure and run the tool, see the [redis-benchmark documentation](https://redis.io/docs/management/optimization/benchmarks/) and the [redis-benchmark examples](#redis-benchmark-examples) sections.
35+
36+
## Benchmarking recommendations
37+
38+
- It's important to not only test the performance of your cache under steady state conditions. *Test under failover conditions too*, and measure the CPU/Server Load on your cache during that time. You can start a failover by [rebooting the primary node](cache-administration.md#reboot). Testing under failover conditions allows you to see the throughput and latency of your application during failover conditions. Failover can happen during updates or during an unplanned event. Ideally, you don't want to see CPU/Server Load peak to more than say 80% even during a failover as that can affect performance.
39+
40+
- Consider using Enterprise and Premium tier Azure Cache for Redis instances. These cache sizes have better network latency and throughput because they're running on better hardware.
41+
42+
- The Enterprise tier generally has the best performance, as Redis Enterprise allows the core Redis process to utilize multiple vCPUs. Tiers based on open source Redis, such as Standard and Premium, are only able to utilize one vCPU for the Redis process per shard.
43+
44+
- Benchmarking the Enterprise Flash tier can be difficult because some keys are stored on DRAM whiles some are stored on a NVMe flash disk. The keys on DRAM benchmark almost as fast as an Enterprise tier instance, but the keys on the NVMe flash disk are slower. Since the Enterprise Flash tier intelligently places the most-used keys into DRAM, ensure that your benchmark configuration matches the actual usage you expect. Consider using the `-r` parameter to randomize which keys are accessed.
45+
46+
- Using TLS/SSL decreases throughput performance, which can be seen clearly in the example benchmarking data in the following tables.
47+
48+
- Even though a Redis server is single-threaded, scaling up tends to improve throughput performance. System processes can use the extra vCPUs instead of sharing the vCPU being used by the Redis process. Scaling up is especially helpful on the Enterprise and Enterprise Flash tiers because Redis Enterprise isn't limited to a single thread. For more information, see [Enterprise tier best practices](cache-best-practices-enterprise-tiers.md#scaling).
49+
50+
- On the Premium tier, scaling out, clustering, is typically recommended before scaling up. Clustering allows Redis server to use more vCPUs by sharding data. Throughput should increase roughly linearly when adding shards in this case.
3251

3352
## Redis-benchmark examples
3453

@@ -53,6 +72,121 @@ Pipelined GET requests with 1k payload:
5372
redis-benchmark -h yourcache.redis.cache.windows.net -a yourAccesskey -t GET -n 1000000 -d 1024 -P 50 -c 50
5473
```
5574

75+
**To test throughput of a Basic, Standard, or Premium tier cache using TLS:**
76+
Pipelined GET requests with 1k payload:
77+
```dos
78+
redis-benchmark -h yourcache.redis.cache.windows.net -p 6380 -a yourAccesskey -t GET -n 1000000 -d 1024 -P 50 -c 50 --tls
79+
```
80+
81+
**To test throughput of an Enterprise or Enterprise Flash cache without TLS using OSS Cluster Mode:**
82+
Pipelined GET requests with 1k payload:
83+
```dos
84+
redis-benchmark -h yourcache.region.redisenterprise.cache.azure.net -p 10000 -a yourAccesskey -t GET -n 1000000 -d 1024 -P 50 -c 50 --cluster
85+
```
86+
87+
## Example performance benchmark data
88+
89+
The following tables show the maximum throughput values that were observed while testing various sizes of Standard, Premium, Enterprise, and Enterprise Flash caches. We used `redis-benchmark` from an IaaS Azure VM against the Azure Cache for Redis endpoint. The throughput numbers are only for GET commands. Typically, SET commands have a lower throughput. These numbers are optimized for throughput. Real-world throughput under acceptable latency conditions may be lower.
90+
91+
The following configuration was used to benchmark throughput:
92+
93+
```dos
94+
redis-benchmark -h yourcache.redis.cache.windows.net -a yourAccesskey -t GET -n 1000000 -d 1024 -P 50 -c 50
95+
```
96+
97+
>[!CAUTION]
98+
>These values aren't guaranteed and there's no SLA for these numbers. We strongly recommend that you should [perform your own performance testing](cache-best-practices-performance.md) to determine the right cache size for your application.
99+
>These numbers might change as we post newer results periodically.
100+
>
101+
102+
### Standard tier
103+
104+
| Instance | Size | vCPUs | Expected network bandwidth (Mbps)| GET requests per second without SSL (1-kB value size) | GET requests per second with SSL (1-kB value size) |
105+
| --- | --- | --- | --- | --- | --- |
106+
| C0 | 250 MB | Shared | 100 | 15,000 | 7,500 |
107+
| C1 | 1 GB | 1 | 500 | 38,000 | 20,720 |
108+
| C2 | 2.5 GB | 2 | 500 | 41,000 | 37,000 |
109+
| C3 | 6 GB | 4 | 1000 | 100,000 | 90,000 |
110+
| C4 | 13 GB | 2 | 500 | 60,000 | 55,000 |
111+
| C5 | 26 GB | 4 | 1,000 | 102,000 | 93,000 |
112+
| C6 | 53 GB | 8 | 2,000 | 126,000 | 120,000 |
113+
114+
### Premium tier
115+
| Instance | Size | vCPUs | Expected network bandwidth (Mbps)| GET requests per second without SSL (1-kB value size) | GET requests per second with SSL (1-kB value size) |
116+
| --- | --- | --- | --- | --- | --- |
117+
| P1 | 6 GB | 2 | 1,500 | 180,000 | 172,000 |
118+
| P2 | 13 GB | 4 | 3,000 | 350,000 | 341,000 |
119+
| P3 | 26 GB | 4 | 3,000 | 350,000 | 341,000 |
120+
| P4 | 53 GB | 8 | 6,000 | 400,000 | 373,000 |
121+
| P5 | 120 GB | 32 | 6,000 | 400,000 | 373,000 |
122+
123+
> [!Important]
124+
> P5 instances in the China East and China North regions use 20 cores, not 32 cores.
125+
126+
### Enterprise & Enterprise Flash tiers
127+
128+
The Enterprise and Enterprise Flash tiers offer a choice of cluster policy: _Enterprise_ and _OSS_. Enterprise cluster policy is a simpler configuration that doesn't require the client to support clustering. OSS cluster policy, on the other hand, uses the [Redis cluster protocol](https://redis.io/docs/management/scaling) to support higher throughputs. We recommend using OSS cluster policy in most cases. For more information, see [Clustering on Enterprise](cache-best-practices-enterprise-tiers.md#clustering-on-enterprise). Benchmarks for both cluster policies are shown in the following tables.
129+
130+
**Enterprise Cluster Policy**
131+
132+
| Instance | Size | vCPUs | Expected network bandwidth (Mbps)| GET requests per second without SSL (1-kB value size) | GET requests per second with SSL (1-kB value size) |
133+
| --- | --- | --- | --- | --- | --- |
134+
| E10 | 12 GB | 4 | 4,000 | 300,000 | 200,000 |
135+
| E20 | 25 GB | 4 | 4,000 | 550,000 | 390,000 |
136+
| E50 | 50 GB | 8 | 8,000 | 950,000 | 530,000 |
137+
| E100 | 100 GB | 16 | 10,000 | 1,300,000 | 580,000 |
138+
| F300 | 384 GB | 8 | 3,200 | 650,000 | 310,000 |
139+
| F700 | 715 GB | 16 | 6,400 | 650,000 | 350,000 |
140+
| F1500 | 1455 GB | 32 | 12,800 | 650,000 | 360,000 |
141+
142+
**OSS Cluster Policy**
143+
144+
| Instance | Size | vCPUs | Expected network bandwidth (Mbps)| GET requests per second without SSL (1-kB value size) | GET requests per second with SSL (1-kB value size) |
145+
| --- | --- | --- | --- | --- | --- |
146+
| E10 | 12 GB | 4 | 4,000 | 1,300,000 | 800,000 |
147+
| E20 | 25 GB | 4 | 4,000 | 1,000,000 | 710,000 |
148+
| E50 | 50 GB | 8 | 8,000 | 2,000,000 | 950,000 |
149+
| E100 | 100 GB | 16 | 10,000 | 2,000,000 | 960,000 |
150+
| F300 | 384 GB | 8 | 3,200 | 1,300,000 | 610,000 |
151+
| F700 | 715 GB | 16 | 6,400 | 1,300,000 | 680,000 |
152+
| F1500 | 1455 GB | 32 | 12,800 | 1,300,000 | 620,000 |
153+
154+
### Enterprise & Enterprise Flash Tiers - Scaled Out
155+
156+
In addition to scaling up by moving to larger cache size, you can boost performance by [scaling out](cache-how-to-scale.md#how-to-scale-up-and-out---enterprise-and-enterprise-flash-tiers). In the Enterprise tiers, scaling out is called increasing the _capacity_ of the cache instance. A cache instance by default has capacity of two--meaning a primary and replica node. An Enterprise cache instance with a capacity of four indicates that the instance was scaled out by a factor of two. Scaling out provides access to more memory and vCPUs. Details on how many vCPUs are used by the core Redis process at each cache size and capacity can be found at the [Enterprise tiers best practices page](cache-best-practices-enterprise-tiers.md#sharding-and-cpu-utilization). Scaling out is most effective when using the OSS cluster policy.
157+
158+
The following tables show the GET requests per second at different capacities, using SSL and a 1-kB value size.
159+
160+
**Scaling out - Enterprise cluster policy**
161+
162+
| Instance | Capacity 2 | Capacity 4 | Capacity 6 |
163+
| --- | --- | --- | --- |
164+
| E10 | 200,000 | 530,000 | 570,000 |
165+
| E20 | 390,000 | 520,000 | 580,000 |
166+
| E50 | 530,000 | 580,000 | 580,000 |
167+
| E100 | 580,000 | 580,000 | 580,000 |
168+
169+
| Instance | Capacity 3 | Capacity 9 |
170+
| --- | --- | --- |
171+
| F300 | 310,000 | 530,000 |
172+
| F700 | 350,000 | 550,000 |
173+
| F1500 | 360,000 | 550,000 |
174+
175+
**Scaling out - OSS cluster policy**
176+
177+
| Instance | Capacity 2 | Capacity 4 | Capacity 6 |
178+
| --- | --- | --- | --- |
179+
| E10 | 800,000 | 720,000 | 1,280,000 |
180+
| E20 | 710,000 | 950,000 | 1,250,000 |
181+
| E50 | 950,000 | 1,260,000 | 1,300,000 |
182+
| E100 | 960,000 | 1,840,000 | 1,930,000|
183+
184+
| Instance | Capacity 3 | Capacity 9 |
185+
| --- | --- | --- |
186+
| F300 | 610,000 | 970,000 |
187+
| F700 | 680,000 | 1,280,000 |
188+
| F1500 | 620,000 | 1,850,000 |
189+
56190
## Next steps
57191

58192
- [Development](cache-best-practices-development.md)

‎articles/azure-cache-for-redis/cache-planning-faq.yml

Lines changed: 1 addition & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -17,41 +17,7 @@ sections:
1717
- question: |
1818
Azure Cache for Redis performance
1919
answer: |
20-
The following table shows the maximum bandwidth values that were observed while testing various sizes of Standard and Premium caches. We used `redis-benchmark.exe` from an IaaS VM against the Azure Cache for Redis endpoint. For TLS throughput, redis-benchmark is used with *stunnel* to connect to the Azure Cache for Redis endpoint.
21-
22-
>[!NOTE]
23-
>These values aren't guaranteed and there's no SLA for these numbers, but should be typical. You should load test your own application to determine the right cache size for your application.
24-
>These numbers might change as we post newer results periodically.
25-
>
26-
27-
From this table, we can draw the following conclusions:
28-
29-
* Throughput numbers are for GETS.
30-
* Throughput for the caches that are the same size is higher in the Premium tier as compared to the Standard tier. For example, with a 6 GB Cache, throughput of P1 is 180,000 requests per second (RPS) as compared to 100,000 RPS for C3.
31-
* With Redis clustering, throughput increases linearly as you increase the number of shards (nodes) in the cluster. For example, if you create a P4 cluster of 10 shards, then the available throughput is 400,000 * 10 = 4 million RPS.
32-
* Throughput for bigger key sizes is higher in the Premium tier as compared to the Standard Tier.
33-
34-
| Pricing tier | Size | CPU cores | Available bandwidth | 1-KB value size | 1-KB value size |
35-
| --- | --- | --- | --- | --- | --- |
36-
| **Standard cache sizes** | | |**Megabits per sec (Mb/s) / Megabytes per sec (MB/s)** |**GET Requests per second Non-SSL** |**GET Requests per second SSL** |
37-
| C0 | 250 MB | Shared | 100 / 12.5 | 15,000 | 7,500 |
38-
| C1 | 1 GB | 1 | 500 / 62.5 | 38,000 | 20,720 |
39-
| C2 | 2.5 GB | 2 | 500 / 62.5 | 41,000 | 37,000 |
40-
| C3 | 6 GB | 4 | 1000 / 125 | 100,000 | 90,000 |
41-
| C4 | 13 GB | 2 | 500 / 62.5 | 60,000 | 55,000 |
42-
| C5 | 26 GB | 4 | 1,000 / 125 | 102,000 | 93,000 |
43-
| C6 | 53 GB | 8 | 2,000 / 250 | 126,000 | 120,000 |
44-
| **Premium cache sizes** | |**CPU cores per shard** | **Megabits per sec (Mb/s) / Megabytes per sec (MB/s)** |**GET Requests per second Non-SSL, per shard** |**GET Requests per second SSL, per shard** |
45-
| P1 | 6 GB | 2 | 1,500 / 187.5 | 180,000 | 172,000 |
46-
| P2 | 13 GB | 4 | 3,000 / 375 | 350,000 | 341,000 |
47-
| P3 | 26 GB | 4 | 3,000 / 375 | 350,000 | 341,000 |
48-
| P4 | 53 GB | 8 | 6,000 / 750 | 400,000 | 373,000 |
49-
| P5 | 120 GB | 32 | 6,000 / 750 | 400,000 | 373,000 |
50-
51-
> [!Important]
52-
> P5 instances in the China East and China North regions use 20 cores, not 32 cores.
53-
54-
For instructions on setting up stunnel or downloading the Redis tools such as `redis-benchmark.exe`, see [How can I run Redis commands?](cache-development-faq.yml#how-can-i-run-redis-commands-).
20+
For guidance on how to benchmark the performance of your Azure Cache for Redis instance and for sample benchmarking numbers, please see the [Performance testing](cache-best-practices-performance.md#example-performance-benchmark-data) page.
5521
5622
- question: |
5723
In what region should I locate my cache?

0 commit comments

Comments
 (0)