Skip to content

Commit 6cca263

Browse files
marcbelmontmarcbelmont
authored
Merge pull request #259 from parkpow/add-new-blur-benchmarks
Added updated and unified blur benchmarks
2 parents eecb6ce + d81cc09 commit 6cca263

File tree

3 files changed

+65
-32
lines changed

3 files changed

+65
-32
lines changed

‎assets/cars-4k.jpg‎

3.66 MB
Loading

‎benchmark/benchmark_blur.md‎

Lines changed: 64 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
The results are obtained using [benchmark_snapshot.py](benchmark_snapshot.py).
44

5-
- The image used is `assets/car-4k.jpg` resized to target resolution.
5+
- The images used are `assets/car-4k.jpg` and `assets/cars-4k.jpg` resized to target resolution.
66
- API is called 50 times for each configuration.
77
- API calls are parallelized. The client code makes **4 calls concurrently**.
88
- Default config with blur enabled by setting **PLATES=10** and **FACES=10**
@@ -17,7 +17,57 @@ python3 -m benchmark.benchmark_snapshot --blur --sdk-url http://localhost:8001
1717
- **Speed** is duration / number_of_calls.
1818
- All numbers are in **milliseconds**.
1919

20-
## AWS Instance Type - t2-large 2-vCPU(2 core) 8 GiB Memory
20+
## Current Version Benchmarks
21+
**PlateRecognizer Blur**: v1.0.8
22+
**PlateRecognizer Snapshot**: v1.54.0
23+
24+
### Local Machine Benchmark - 12th Gen Intel Core i3-1215U (6 Cores, 8 Threads, 16 GiB RAM)
25+
Processor: Intel Core i3-1215U @ 1.20GHz
26+
Manufacturer: GenuineIntel
27+
NumCores: 6
28+
NumLogicals: 8
29+
CPUFrequency: 4400
30+
Passmark: 10432
31+
32+
#### WORKERS = 1
33+
Image: `assets/car-4k.jpg`
34+
35+
| Resolution | Speed | Peak CPU% (Blur, Snapshot) | Peak MEM in MiB (Blur, Snapshot) |
36+
|------------|-------|----------------------------|----------------------------------|
37+
| 800x600 | 185.2 | 57.04, 7.51 | 529.71, 508.80 |
38+
| 1280x720 | 180.4 | 45.23, 8.20 | 426.93, 510.40 |
39+
| 1920x1080 | 241.5 | 46.40, 9.40 | 430.35, 520.90 |
40+
| 2560x1440 | 353.3 | 45.86, 8.45 | 489.37, 529.97 |
41+
| 3840x2160 | 598.4 | 60.96, 8.65 | 510.27, 580.17 |
42+
43+
44+
#### WORKERS = 1
45+
Image: `assets/cars-4k.jpg`
46+
47+
| Resolution | Speed | Peak CPU% (Blur, Snapshot) | Peak MEM in MiB (Blur, Snapshot) |
48+
|------------|-------|----------------------------|----------------------------------|
49+
| 800x600 | 388.0 | 45.84, 7.52 | 289.65, 338.73 |
50+
| 1280x720 | 377.6 | 40.62, 9.11 | 346.34, 340.29 |
51+
| 1920x1080 | 400.1 | 41.80, 9.82 | 295.51, 350.61 |
52+
| 2560x1440 | 447.4 | 42.29, 10.08 | 303.67, 370.47 |
53+
| 3840x2160 | 631.3 | 54.01, 8.31 | 375.21, 410.12 |
54+
55+
56+
#### WORKERS = 2
57+
Image: `assets/cars-4k.jpg`
58+
59+
| Resolution | Speed | Peak CPU% (Blur, Snapshot) | Peak MEM in MiB (Blur, Snapshot) |
60+
|------------|-------|----------------------------|----------------------------------|
61+
| 800x600 | 744.0 | 45.21, 8.79 | 396.69, 338.70 |
62+
| 1280x720 | 742.7 | 40.71, 9.52 | 359.91, 338.91 |
63+
| 1920x1080 | 779.9 | 40.22, 9.63 | 323.12, 350.52 |
64+
| 2560x1440 | 878.3 | 43.36, 10.31 | 372.47, 370.47 |
65+
| 3840x2160 | 1237.2 | 52.9, 8.71 | 375.41, 410.03 |
66+
67+
68+
## Previous Version Benchmarks
69+
70+
### AWS Instance Type - t2-large 2-vCPU(2 core) 8 GiB Memory
2171
Processor: Intel Xeon CPU E5-2686 v4 @ 2.30GHz
2272
NumSockets: 1
2373
Manufacturer: GenuineIntel
@@ -26,8 +76,8 @@ NumLogicals: 2
2676
CPUFrequency: 2300
2777
Passmark: 2420.9990
2878

29-
### WORKERS = 1 (DEFAULT)
30-
79+
#### WORKERS = 1
80+
Image: `assets/car-4k.jpg`
3181

3282
| Resolution | Speed | Peak CPU% (Blur, Snapshot) | Peak MEM in MiB (Blur, Snapshot) |
3383
|------------|-------|----------------------------|----------------------------------|
@@ -36,8 +86,8 @@ Passmark: 2420.9990
3686
| 1920x1080 | 528.0 | 115.30, 46.70 | 692.35, 236.80 |
3787
| 2560x1440 | 975.7 | 112.86, 46.75 | 746.27, 235.77 |
3888

39-
### WORKERS = 2
40-
89+
#### WORKERS = 2
90+
Image: `assets/car-4k.jpg`
4191

4292
| Resolution | Speed | Peak CPU% (Blur, Snapshot) | Peak MEM in MiB (Blur, Snapshot) |
4393
|------------|-------|----------------------------|----------------------------------|
@@ -46,7 +96,7 @@ Passmark: 2420.9990
4696
| 1920x1080 | 435.9 | 149.61, 61.38 | 781.14, 390.07 |
4797
| 2560x1440 | 786.3 | 194.81, 83.06 | 868.37, 441.07 |
4898

49-
## AWS Instance Type - t3-xlarge 4-vCPU(2 core) 16 GiB Memory
99+
### AWS Instance Type - t3-xlarge 4-vCPU(2 core) 16 GiB Memory
50100
Processor: Intel Xeon Platinum 8259CL @ 2.50GHz
51101
NumSockets: 1
52102
Manufacturer: GenuineIntel
@@ -55,7 +105,8 @@ NumLogicals: 4
55105
CPUFrequency: 2499
56106
Passmark: 4039.57
57107

58-
### WORKERS = 1 (DEFAULT)
108+
#### WORKERS = 1
109+
Image: `assets/car-4k.jpg`
59110

60111
| Resolution | Speed | Peak CPU% (Blur, Snapshot) | Peak MEM in MiB (Blur, Snapshot) |
61112
|------------|-------|----------------------------|----------------------------------|
@@ -64,7 +115,8 @@ Passmark: 4039.57
64115
| 1920x1080 | 519.1 | 212.69,57.34 | 497.72, 227.89 |
65116
| 2560x1440 | 964.2 | 209.77, 47.42 | 568.27, 238.00 |
66117

67-
### WORKERS = 2
118+
#### WORKERS = 2
119+
Image: `assets/car-4k.jpg`
68120

69121
| Resolution | Speed | Peak CPU% (Blur, Snapshot) | Peak MEM in MiB (Blur, Snapshot) |
70122
|------------|-------|----------------------------|----------------------------------|
@@ -73,29 +125,12 @@ Passmark: 4039.57
73125
| 1920x1080 | 387.6 | 303.41, 83.80 | 1297.73, 404.14 |
74126
| 2560x1440 | 717.5 | 338.79, 104.97 | 1362.45, 448.89 |
75127

76-
### WORKERS = 4
128+
#### WORKERS = 4
129+
Image: `assets/car-4k.jpg`
77130

78131
| Resolution | Speed | Peak CPU% (Blur, Snapshot) | Peak MEM in MiB (Blur, Snapshot) |
79132
|------------|-------|----------------------------|----------------------------------|
80133
| 800x600 | 295.1 | 378.88, 41.19 | 1665.84, 661.40 |
81134
| 1280x720 | 291.8 | 396.42, 68.414 | 1630.47, 690.65 |
82135
| 1920x1080 | 360.1 | 371.16, 93.14 | 1711.66, 715.22 |
83-
| 2560x1440 | 614.7 | 402.30, 164.91 | 1800.31, 790.79 |
84-
85-
## Local Machine Benchmark - 12th Gen Intel Core i3-1215U (6 Cores, 8 Threads, 16 GiB RAM)
86-
Processor: Intel Core i3-1215U @ 1.20GHz
87-
Manufacturer: GenuineIntel
88-
NumCores: 6
89-
NumLogicals: 8
90-
CPUFrequency: 4400
91-
Passmark: 10432
92-
93-
### 4K Images Benchmark
94-
95-
| Filename | Avg Speed (ms) | Min Speed (ms) | Max Speed (ms) |
96-
|---------------|----------------|----------------|----------------|
97-
| [single-1.jpg](https://drive.google.com/file/d/19IafPgGJLawOn7fv4MbXFd1Wc4yt78-X/view?usp=drive_link) | 1920.3 | 559.4 | 2129.1 |
98-
| [single-2.jpg](https://drive.google.com/file/d/1ntqdpxi4cDSkTr14RaqUlKC8f9wnCs2M/view?usp=drive_link) | 2442.6 | 691.4 | 2619.9 |
99-
| [many-1.jpg](https://drive.google.com/file/d/126AphdZb6jqnUVjChmZRiRCdIk_1FuGN/view?usp=drive_link) | 3266.8 | 817.3 | 3965.1 |
100-
| [many-2.jpg](https://drive.google.com/file/d/16hfH_nGZ_omyfiQtT_U69NI3Ju5jQ3Aw/view?usp=drive_link) | 4514.9 | 1092.2 | 5180.0 |
101-
| [many-3.jpg](https://drive.google.com/file/d/1_r1w6Z5K9IJUWJJe9mzifpwZNU_Li9M0/view?usp=drive_link) | 3687.1 | 989.0 | 4175.9 |
136+
| 2560x1440 | 614.7 | 402.30, 164.91 | 1800.31, 790.79 |

‎benchmark/benchmark_snapshot.py‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -78,15 +78,14 @@ def call_duration(path, sdk_url, config, mmc, blur):
7878

7979
def benchmark(args, executor):
8080
image = Image.open(args.image)
81-
for resolution in [(800, 600), (1280, 720), (1920, 1080), (2560, 1440)]:
81+
for resolution in [(800, 600), (1280, 720), (1920, 1080), (2560, 1440), (3840, 2160)]:
8282
image.resize(resolution).save("/tmp/platerec-benchmark.jpg")
8383
if args.blur:
8484
configs = [{}]
8585
else:
8686
configs = [{}, dict(mode="fast")]
8787

8888
for config in configs:
89-
now = default_timer()
9089
stats = list(
9190
executor.map(
9291
partial(
@@ -99,7 +98,6 @@ def benchmark(args, executor):
9998
["/tmp/platerec-benchmark.jpg"] * args.iterations,
10099
)
101100
)
102-
duration = (default_timer() - now) * 1000
103101
yield dict(
104102
resolution="%sx%s" % resolution,
105103
mode=config.get("mode", "regular"),

0 commit comments

Comments
 (0)