Skip to content

Commit cabc654

Browse files
semihalevsemihalev
authored
release: 1.8.0 — version marks, BENCHMARKS.md, README pointer (#573)
* docs: add BENCHMARKS.md — method, configurations, and measured results * docs, version: 1.8.0 everywhere, README points at BENCHMARKS.md The README's June-2025 benchmark section is replaced by a pointer to BENCHMARKS.md, which carries the current method, configurations and results with their caveats. The binary version and the docker example tag move from 1.8.0-rc2 to 1.8.0. * docs: 1.8.x is the supported security line
1 parent 8b36b91 commit cabc654

4 files changed

Lines changed: 132 additions & 37 deletions

File tree

‎BENCHMARKS.md‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
# Benchmarks
2+
3+
Measured 2026-08-18 at commit `8b36b91` (the 1.8.0 serving-path work, PR #572).
4+
This document exists to make one set of claims precisely, with the method and
5+
configurations needed to check them — not to advertise a bigger number than the
6+
method supports.
7+
8+
**What these numbers are:** the cached-answer serving ceiling of each resolver
9+
on one machine — how fast the server itself can answer once the answer is in
10+
its cache. **What they are not:** a prediction of production throughput. Real
11+
traffic mixes hits with misses, and a miss is bound by upstream latency, not by
12+
the serving engine. What a resolver's engine controls is the hit path; that is
13+
what this measures.
14+
15+
## Environment
16+
17+
| | |
18+
|---|---|
19+
| Host | 2× Intel Xeon E5-2620 v4 @ 2.10 GHz (32 logical cores), 64 GB RAM |
20+
| OS | Ubuntu 26.04 LTS, stock kernel and sysctls — no network tuning |
21+
| Load generator | dnsperf 2.15.0, on the same host over loopback |
22+
| sdns build | Go 1.26.5, `go build`, no build flags |
23+
24+
Client and server share the machine, so every number includes the load
25+
generator's own CPU cost, identically for every contender.
26+
27+
## Method
28+
29+
- **Corpus:** 2,349 names pre-verified to be served from cache (plus separate
30+
corpora: 1,143 names covered by cached negative answers, 63 names with
31+
cached SERVFAIL). Warm-hit corpora decay as TTLs expire, so every
32+
measurement is preceded by a fresh warm pass; numbers taken against a stale
33+
cache measure recursion, not serving, and come out far lower.
34+
- **Protocol per contender:** start fresh → warm the corpus (parallel `dig`
35+
pass) → one 10-second throwaway run → the measured runs, 20 seconds each.
36+
Median and best of the series are reported.
37+
- **UDP load shape:** `dnsperf -c 128 -T 8 -l 20`. The flow count matters: with
38+
only 20 flows (`-c 20`), kernel reuseport hashing leaves most of a 32-socket
39+
receiver idle and the results measure hash luck. 128 flows approximates real
40+
traffic, which carries thousands. (For reference, the 20-flow shape puts
41+
sdns and PowerDNS at parity around 370–400k and does not change the ordering
42+
of the others.)
43+
- **TCP load shape:** `dnsperf -m tcp -c 20 -T 4 -l 20`, pipelined persistent
44+
connections.
45+
- **DNSSEC validation enabled in all four resolvers** (AD flag spot-checked
46+
through each). IPv6 upstream disabled everywhere (the host has no v6
47+
transit); irrelevant to cached serving.
48+
49+
## Contenders
50+
51+
| Resolver | Version | Serving configuration |
52+
|---|---|---|
53+
| sdns | 1.8.0 @ `8b36b91` | stock generated config (bind/API/paths only); full middleware chain runs per query |
54+
| PowerDNS Recursor | 5.4.1 | `threads=8`, `reuseport=yes`, `dnssec=validate`; warm hits served by the packet cache |
55+
| Unbound | 1.24.2 | `num-threads: 8`, `so-reuseport: yes`, cache slabs = 8, `msg-cache-size: 256m`, `rrset-cache-size: 512m`, `minimal-responses: yes` |
56+
| Knot Resolver | 6.2.0 | 8 `kresd` instances on one port (SO_REUSEPORT), shared 512 MB LMDB cache |
57+
58+
Two fairness notes, one in each direction. PowerDNS's packet cache echoes a
59+
stored packet — deliberately less work per query than sdns's full chain, so
60+
its number represents its lightest possible path, as does ours. And each
61+
contender was given a reasonable performance configuration, not an exhaustive
62+
tuning pass; a specialist could likely move any of these numbers some percent.
63+
64+
## Results
65+
66+
### UDP, cached answers (`-c 128 -T 8`, 3×20 s)
67+
68+
| Resolver | median qps | best qps |
69+
|---|---|---|
70+
| **sdns 1.8.0** | **424k** | **444k** |
71+
| PowerDNS Recursor 5.4.1 | 371k | 390k |
72+
| Unbound 1.24.2 | 343k | 346k |
73+
| Knot Resolver 6.2.0 | 191k | 192k |
74+
75+
sdns with the untouched default configuration measures in the same band
76+
(median 406k over three runs) — the result does not depend on tuning knobs.
77+
Answer classes beyond plain hits, measured on sdns freshly warmed: negative
78+
answers (NXDOMAIN from cached denial) 409k, cached SERVFAIL 399k.
79+
80+
### TCP, cached answers (`-c 20 -T 4`, 5×20 s)
81+
82+
| Resolver | median qps | best qps |
83+
|---|---|---|
84+
| **sdns 1.8.0** | **226k** | **273k** |
85+
| Knot Resolver 6.2.0 | 142k | 146k |
86+
| Unbound 1.24.2 | 136k | 149k |
87+
| PowerDNS Recursor 5.4.1 | 56k | 57k |
88+
89+
### Run-to-run spread
90+
91+
20-second runs on a busy OS have real variance; the full series behind the
92+
medians spanned roughly ±7% for sdns UDP (423–444k), ±6% for PowerDNS
93+
(346–390k), ±3% for Unbound, ±2% for Knot, and ±15% for sdns TCP (195–273k).
94+
Single-run numbers from any resolver should be read with that in mind.
95+
96+
## What changed in 1.8.0
97+
98+
The same harness, applied to sdns itself across the 1.8.0 serving-path work
99+
(each row A/B-measured against its predecessor at the time; early rows used
100+
the 20-flow shape, so rows are comparable to their neighbors, not across the
101+
whole column):
102+
103+
| build | UDP cached answers |
104+
|---|---|
105+
| 1.8.0 baseline before PR #572 | 268k |
106+
| + sharded slab caches | 287k |
107+
| + fetch-add lease admission | 306k |
108+
| + batch-slot persistence | ~330k |
109+
| + inline wire-hit serving on the reader | 424k median / 444k best |
110+
111+
TCP moved from ~100k to the 226k median above in the same PR, by removing a
112+
per-connection query budget that forced a reconnect storm under pipelining.
113+
114+
## Reproducing
115+
116+
```sh
117+
# corpus: one name per line; verify each serves from cache before trusting it
118+
dnsperf -s <addr> -p <port> -d hits.txt -c 128 -T 8 -l 20 # UDP
119+
dnsperf -s <addr> -p <port> -m tcp -d hits.txt -c 20 -T 4 -l 20 # TCP
120+
```
121+
122+
Warm first, discard a throwaway run, take at least three measurements, report
123+
the median, and state the flow count — it is the parameter that moves these
124+
numbers the most.

‎README.md‎

Lines changed: 5 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ $ docker run -d --name sdns -p 53:53 -p 53:53/udp ghcr.io/semihalev/sdns:latest
4545
Pin to a specific version (recommended for production):
4646

4747
```shell
48-
$ docker run -d --name sdns -p 53:53 -p 53:53/udp ghcr.io/semihalev/sdns:1.8.0-rc2
48+
$ docker run -d --name sdns -p 53:53 -p 53:53/udp ghcr.io/semihalev/sdns:1.8.0
4949
```
5050

5151
#### Docker Compose
@@ -572,39 +572,10 @@ This is useful when:
572572
573573
## Performance
574574
575-
### Benchmark Environment
576-
577-
* **Server Specifications:**
578-
* Processor: Apple M1 Pro
579-
* Memory: 16GB
580-
581-
### Benchmarking Tool
582-
583-
* **Tool:** [DNS-OARC dnsperf](https://www.dns-oarc.net/tools/dnsperf)
584-
* **Configuration:**
585-
* Query volume: 50,000 sample queries
586-
* Test date: June 2025
587-
588-
### Benchmark Comparisons
589-
590-
Tests were performed on the following DNS resolvers: SDNS 1.6.5, PowerDNS Recursor 5.4.1, BIND 9.19.12, and Unbound 1.17.1.
591-
592-
### Benchmark Results
593-
594-
| Resolver | Version | QPS | Avg Latency | Lost Queries | Runtime | Response Codes |
595-
| -------- | ------- | ------ | ----------- | ------------ | -------- | --------------------------------------------------- |
596-
| SDNS | 1.6.5 | 708/s | 134ms | 1 (0.00%) | 70.5s | NOERROR: 66.87%, SERVFAIL: 1.71%, NXDOMAIN: 31.43% |
597-
| PowerDNS | 5.4.1 | 617/s | 147ms | 17 (0.03%) | 80.9s | NOERROR: 66.87%, SERVFAIL: 1.69%, NXDOMAIN: 31.44% |
598-
| BIND | 9.19.12 | 405/s | 200ms | 156 (0.31%) | 123.0s | NOERROR: 67.84%, SERVFAIL: 1.62%, NXDOMAIN: 30.54% |
599-
| Unbound | 1.17.1 | 338/s | 237ms | 263 (0.53%) | 147.0s | NOERROR: 68.20%, SERVFAIL: 1.20%, NXDOMAIN: 30.60% |
600-
601-
### Performance Summary
602-
603-
SDNS demonstrates superior performance across all key metrics:
604-
- **Highest throughput**: 708 queries per second (15% faster than PowerDNS, 75% faster than BIND, 109% faster than Unbound)
605-
- **Lowest latency**: 134ms average (9-43% lower than competitors)
606-
- **Best reliability**: Only 1 lost query out of 50,000 (99.998% success rate)
607-
- **Fastest completion**: 70.5 seconds total runtime
575+
Throughput measurements, methodology, resolver comparisons and their caveats
576+
live in [BENCHMARKS.md](BENCHMARKS.md) — measured with dnsperf against
577+
PowerDNS Recursor, Unbound and Knot Resolver under identical load, most
578+
recently for the 1.8.0 serving-path work.
608579
609580
For Kubernetes DNS, the registry is the hot path:
610581

‎SECURITY.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,8 @@ Only the latest minor release line receives security updates:
66

77
| Version | Supported |
88
| ------- | ------------------ |
9-
| 1.7.x | :white_check_mark: |
10-
| < 1.7 | :x: |
9+
| 1.8.x | :white_check_mark: |
10+
| < 1.8 | :x: |
1111

1212
## Reporting a Vulnerability
1313

‎sdns.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ import (
2121
"github.com/spf13/cobra"
2222
)
2323

24-
const version = "1.8.0-rc2"
24+
const version = "1.8.0"
2525

2626
var (
2727
cfgPath string

0 commit comments

Comments
 (0)