Skip to content

Commit 6707e5a

Browse files
ciarams87ciarams87
authored
Add dataplane performance test (#1140)
* Add dataplane performance test
1 parent 6e1c9d8 commit 6707e5a

File tree

7 files changed

+332
-0
lines changed

7 files changed

+332
-0
lines changed

‎tests/dataplane-performance/manifests/cafe-routes.yaml‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
apiVersion: gateway.networking.k8s.io/v1beta1
2+
kind: HTTPRoute
3+
metadata:
4+
name: coffee
5+
spec:
6+
parentRefs:
7+
- name: cafe
8+
hostnames:
9+
- cafe.example.com
10+
rules:
11+
- matches:
12+
- path:
13+
type: PathPrefix
14+
value: /latte
15+
backendRefs:
16+
- name: coffee-svc
17+
port: 80
18+
- matches:
19+
- path:
20+
type: PathPrefix
21+
value: /coffee
22+
headers:
23+
- name: version
24+
value: v2
25+
- path:
26+
type: PathPrefix
27+
value: /coffee
28+
queryParams:
29+
- name: TEST
30+
value: v2
31+
backendRefs:
32+
- name: coffee-svc
33+
port: 80
34+
---
35+
apiVersion: gateway.networking.k8s.io/v1beta1
36+
kind: HTTPRoute
37+
metadata:
38+
name: tea
39+
spec:
40+
parentRefs:
41+
- name: cafe
42+
hostnames:
43+
- cafe.example.com
44+
rules:
45+
- matches:
46+
- path:
47+
type: PathPrefix
48+
value: /tea
49+
method: POST
50+
backendRefs:
51+
- name: coffee-svc
52+
port: 80
53+
- matches:
54+
- path:
55+
type: PathPrefix
56+
value: /tea
57+
method: GET
58+
backendRefs:
59+
- name: coffee-svc
60+
port: 80

‎tests/dataplane-performance/manifests/coffee.yaml‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: coffee
5+
spec:
6+
replicas: 1
7+
selector:
8+
matchLabels:
9+
app: coffee
10+
template:
11+
metadata:
12+
labels:
13+
app: coffee
14+
spec:
15+
containers:
16+
- name: coffee
17+
image: nginxdemos/nginx-hello:plain-text
18+
ports:
19+
- containerPort: 8080
20+
---
21+
apiVersion: v1
22+
kind: Service
23+
metadata:
24+
name: coffee-svc
25+
spec:
26+
ports:
27+
- port: 80
28+
targetPort: 8080
29+
protocol: TCP
30+
name: http
31+
selector:
32+
app: coffee

‎tests/dataplane-performance/manifests/gateway.yaml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
apiVersion: gateway.networking.k8s.io/v1beta1
2+
kind: Gateway
3+
metadata:
4+
name: cafe
5+
spec:
6+
gatewayClassName: nginx
7+
listeners:
8+
- name: http
9+
port: 80
10+
protocol: HTTP

‎tests/dataplane-performance/results/1.0.0/1.0.0.md‎

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
# Results
2+
3+
## Test environment
4+
5+
GKE cluster:
6+
7+
- Node count: 3
8+
- Instance Type: e2-medium
9+
- k8s version: 1.27.4-gke.900
10+
- Zone: europe-west2-b
11+
- Total vCPUs: 6
12+
- Total RAM: 12GB
13+
- Max pods per node: 110
14+
15+
Test VM:
16+
17+
- Instance Type: e2-medium
18+
- Zone: europe-west2-b
19+
- vCPUS: 2
20+
- RAM: 4GB
21+
22+
NGF deployment:
23+
24+
- NGF version: edge - git commit a41e9e46d72788bceea8ba44e01d4062afec75fd
25+
- NGINX Version: 1.25.2
26+
27+
## Test1: Running latte path based routing
28+
29+
```console
30+
Running 30s test @ http://cafe.example.com/latte
31+
2 threads and 10 connections
32+
Thread Stats Avg Stdev Max +/- Stdev
33+
Latency 6.29ms 7.85ms 182.06ms 88.67%
34+
Req/Sec 1.11k 588.11 2.31k 59.83%
35+
Latency Distribution
36+
50% 3.44ms
37+
75% 7.09ms
38+
90% 15.33ms
39+
99% 31.86ms
40+
66017 requests in 30.02s, 23.42MB read
41+
Requests/sec: 2199.43
42+
Transfer/sec: 799.00KB
43+
```
44+
45+
## Test2: Running coffee header based routing
46+
47+
```console
48+
Running 30s test @ http://cafe.example.com/coffee
49+
2 threads and 10 connections
50+
Thread Stats Avg Stdev Max +/- Stdev
51+
Latency 31.95ms 37.10ms 243.16ms 89.07%
52+
Req/Sec 224.25 238.65 1.83k 90.32%
53+
Latency Distribution
54+
50% 14.31ms
55+
75% 43.74ms
56+
90% 76.70ms
57+
99% 171.95ms
58+
12913 requests in 30.03s, 4.59MB read
59+
Requests/sec: 430.05
60+
Transfer/sec: 156.65KB
61+
```
62+
63+
## Test3: Running coffee query based routing
64+
65+
```console
66+
Running 30s test @ http://cafe.example.com/coffee?TEST=v2
67+
2 threads and 10 connections
68+
Thread Stats Avg Stdev Max +/- Stdev
69+
Latency 29.26ms 43.85ms 457.05ms 82.26%
70+
Req/Sec 632.30 464.79 2.07k 60.43%
71+
Latency Distribution
72+
50% 4.45ms
73+
75% 45.84ms
74+
90% 103.14ms
75+
99% 145.53ms
76+
37324 requests in 30.02s, 13.56MB read
77+
Requests/sec: 1243.13
78+
Transfer/sec: 462.53KB
79+
```
80+
81+
## Test4: Running tea GET method based routing
82+
83+
```console
84+
Running 30s test @ http://cafe.example.com/tea
85+
2 threads and 10 connections
86+
Thread Stats Avg Stdev Max +/- Stdev
87+
Latency 27.19ms 39.40ms 182.48ms 81.49%
88+
Req/Sec 706.49 416.90 1.85k 65.42%
89+
Latency Distribution
90+
50% 4.06ms
91+
75% 42.24ms
92+
90% 99.21ms
93+
99% 133.34ms
94+
41265 requests in 30.04s, 14.56MB read
95+
Requests/sec: 1373.59
96+
Transfer/sec: 496.31KB
97+
```
98+
99+
## Test5: Running tea POST method based routing
100+
101+
```console
102+
Running 30s test @ http://cafe.example.com/tea
103+
2 threads and 10 connections
104+
Thread Stats Avg Stdev Max +/- Stdev
105+
Latency 27.32ms 39.49ms 208.66ms 81.40%
106+
Req/Sec 685.47 491.33 2.08k 58.43%
107+
Latency Distribution
108+
50% 4.19ms
109+
75% 42.40ms
110+
90% 99.38ms
111+
99% 133.25ms
112+
40723 requests in 30.03s, 14.37MB read
113+
Requests/sec: 1356.10
114+
Transfer/sec: 489.99KB
115+
```
116+
117+
## Observations
118+
119+
- Path based routing is the most performant of the routing methods
120+
- The njs based methods are less performant:
121+
- Header based routing is the least performant of the routing methods, by a factor of roughly 5 compared to the path
122+
based routing.
123+
- The query and method based routing methods are approximately 2/3s as performant as the path based routing.

‎tests/dataplane-performance/scripts/post.lua‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
wrk.method = "POST"

‎tests/dataplane-performance/scripts/wrk-latency.sh‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
#!/bin/bash
2+
3+
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
4+
5+
echo -e "# Results\n"
6+
7+
echo -e "## Test1: Running latte path based routing\n"
8+
echo -e '```console'
9+
wrk -t2 -c10 -d30 http://cafe.example.com/latte --latency
10+
echo -e '```'
11+
echo -e "\n## Test2: Running coffee header based routing\n"
12+
echo -e '```console'
13+
wrk -t2 -c10 -d30 http://cafe.example.com/coffee -H "version: v2" --latency
14+
echo -e '```'
15+
echo -e "\n## Test3: Running coffee query based routing\n"
16+
echo -e '```console'
17+
wrk -t2 -c10 -d30 http://cafe.example.com/coffee?TEST=v2 --latency
18+
echo -e '```'
19+
echo -e "\n## Test4: Running tea GET method based routing\n"
20+
echo -e '```console'
21+
wrk -t2 -c10 -d30 http://cafe.example.com/tea --latency
22+
echo -e '```'
23+
echo -e "\n## Test5: Running tea POST method based routing\n"
24+
echo -e '```console'
25+
wrk -t2 -c10 -d30 http://cafe.example.com/tea -s ${SCRIPT_DIR}/post.lua --latency
26+
echo -e '```'

‎tests/dataplane-performance/setup.md‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Dataplane Performance Testing
2+
3+
## Goals
4+
5+
- To capture the average latency of requests being proxied through NGINX using a variety of routing rules, so that we
6+
can see if different routing rules provide different results, and so that we can know if any future work has an impact
7+
on data plane performance.
8+
- A route is created and tested for each routing method below:
9+
- path based routing
10+
- header based routing
11+
- query param based routing
12+
- method based routing
13+
14+
## Test Environment
15+
16+
- A Kubernetes cluster with 3 nodes on GKE
17+
- Node: e2-medium (2 vCPU, 4GB memory)
18+
- Tester VM on Google Cloud:
19+
- Instance Type: e2-medium (2 vCPU, 4GB memory)
20+
- Configuration:
21+
- Debian
22+
- Install packages: wrk
23+
- Location - same zone as the Kubernetes cluster.
24+
25+
## Setup
26+
27+
1. Create cloud cluster
28+
2. Deploy CRDs:
29+
30+
```bash
31+
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v0.8.1/standard-install.yaml
32+
```
33+
34+
3. Deploy NGF from edge using Helm install, and expose using an internal LoadBalancer service:
35+
36+
```console
37+
helm install my-release oci://ghcr.io/nginxinc/charts/nginx-gateway-fabric --version 0.0.0-edge \
38+
--set service.annotations.'networking\.gke\.io\/load-balancer-type'="Internal" --create-namespace \
39+
--wait -n nginx-gateway
40+
```
41+
42+
## Tests
43+
44+
1. First create the test resources. The manifests provided will deploy a single application and service, and two
45+
HTTPRoutes with matches for path based, header based, query based, and method based routing.
46+
47+
```console
48+
kubectl apply -f manifests/gateway.yaml
49+
kubectl apply -f manifests/coffee.yaml
50+
kubectl apply -f manifests/cafe-routes.yaml
51+
```
52+
53+
2. Get the external IP of the nginx-gateway service and add an entry in /etc/hosts for `<GW_IP> cafe.example.com`.
54+
55+
```console
56+
kubectl get service my-release-nginx-gateway-fabric \
57+
--namespace nginx-gateway \
58+
--output jsonpath='{.status.loadBalancer.ingress[0].ip}'
59+
```
60+
61+
3. Copy the scripts in the `scripts/` directory to the test VM. Run tests using `wrk` (change the file output name to
62+
version under test). The tests use `wrk` to send requests to coffee application using each of the configured routing
63+
rules, capturing the number of requests and the average latency.
64+
65+
```console
66+
bash wrk-latency.sh > 1.0.0.md
67+
```
68+
69+
4. Analyse the results and check for anomolies. Copy the results file from the test VM into the `results/` directory.
70+
Append any findings or observations to the generated results document. Add test environment information to the
71+
generated document.
72+
73+
5. Cleanup the deployed resources
74+
75+
```console
76+
kubectl delete -f manifests/cafe-routes.yaml
77+
kubectl delete -f manifests/coffee.yaml
78+
kubectl delete -f manifests/gateway.yaml
79+
helm uninstall my-release -n nginx-gateway
80+
```

0 commit comments

Comments
 (0)