Skip to content

Commit b47b56f

Browse files
mr-fatalystmr-fatalyst
committed
added benchmarks and results
1 parent b35e0e3 commit b47b56f

File tree

42 files changed

+2137
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+2137
-0
lines changed

‎README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -232,6 +232,12 @@ Examples of integration and detailed usage for each framework are available in t
232232

233233
---
234234

235+
## 📊 Quick & Dirty Benchmarks
236+
237+
Fast but not perfect benchmarks. Check the [`benchmarks`](https://github.com/mr-fatalyst/fastopenapi/tree/master/benchmarks) directory for details.
238+
239+
---
240+
235241
## ✅ Development Recommendations
236242

237243
- Use Pydantic models for strict typing and data validation.

‎benchmarks/FALCON.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Falcon Benchmark
2+
3+
---
4+
5+
## Testing Original Implementation
6+
```
7+
Original - Running 10000 iterations per endpoint
8+
--------------------------------------------------
9+
GET all records: 8.6085 sec total, 0.86 ms per request
10+
GET one record: 8.7683 sec total, 0.88 ms per request
11+
POST new record: 9.1847 sec total, 0.92 ms per request
12+
PUT record: 9.3219 sec total, 0.93 ms per request
13+
PATCH record: 9.1856 sec total, 0.92 ms per request
14+
DELETE record: 17.7793 sec total, 1.78 ms per request
15+
```
16+
---
17+
18+
## Testing FastOpenAPI Implementation
19+
20+
```
21+
FastOpenAPI - Running 10000 iterations per endpoint
22+
--------------------------------------------------
23+
GET all records: 8.8547 sec total, 0.89 ms per request
24+
GET one record: 9.0380 sec total, 0.90 ms per request
25+
POST new record: 9.5613 sec total, 0.96 ms per request
26+
PUT record: 9.5503 sec total, 0.96 ms per request
27+
PATCH record: 9.4769 sec total, 0.95 ms per request
28+
DELETE record: 18.6130 sec total, 1.86 ms per request
29+
```
30+
31+
---
32+
33+
## Performance Comparison (10000 iterations)
34+
35+
| Endpoint | Original | FastOpenAPI | Difference |
36+
|-------------------------|----------|-------------|----------------|
37+
| GET all records | 0.86 ms | 0.89 ms | 0.02 ms (+2.9%) |
38+
| GET one record | 0.88 ms | 0.90 ms | 0.03 ms (+3.1%) |
39+
| POST new record | 0.92 ms | 0.96 ms | 0.04 ms (+4.1%) |
40+
| PUT record | 0.93 ms | 0.96 ms | 0.02 ms (+2.5%) |
41+
| PATCH record | 0.92 ms | 0.95 ms | 0.03 ms (+3.2%) |
42+
| DELETE record | 1.78 ms | 1.86 ms | 0.08 ms (+4.7%) |
43+
44+
---
45+
46+
[<< Back](README.md)
47+
48+
---

‎benchmarks/FLASK.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Flask Benchmark
2+
3+
---
4+
5+
## Testing Original Implementation
6+
```
7+
Original - Running 10000 iterations per endpoint
8+
--------------------------------------------------
9+
GET all records: 12.3051 sec total, 1.23 ms per request
10+
GET one record: 12.5205 sec total, 1.25 ms per request
11+
POST new record: 13.2638 sec total, 1.33 ms per request
12+
PUT record: 13.3373 sec total, 1.33 ms per request
13+
PATCH record: 13.1245 sec total, 1.31 ms per request
14+
DELETE record: 25.7178 sec total, 2.57 ms per request
15+
```
16+
---
17+
18+
## Testing FastOpenAPI Implementation
19+
20+
```
21+
FastOpenAPI - Running 10000 iterations per endpoint
22+
--------------------------------------------------
23+
GET all records: 12.7424 sec total, 1.27 ms per request
24+
GET one record: 13.0017 sec total, 1.30 ms per request
25+
POST new record: 13.5450 sec total, 1.35 ms per request
26+
PUT record: 13.5281 sec total, 1.35 ms per request
27+
PATCH record: 13.5548 sec total, 1.36 ms per request
28+
DELETE record: 26.7906 sec total, 2.68 ms per request
29+
```
30+
31+
---
32+
33+
## Performance Comparison (10000 iterations)
34+
35+
| Endpoint | Original | FastOpenAPI | Difference |
36+
|-------------------------|----------|-------------|----------------|
37+
| GET all records | 1.23 ms | 1.27 ms | 0.04 ms (+3.6%) |
38+
| GET one record | 1.25 ms | 1.30 ms | 0.05 ms (+3.8%) |
39+
| POST new record | 1.33 ms | 1.35 ms | 0.03 ms (+2.1%) |
40+
| PUT record | 1.33 ms | 1.35 ms | 0.02 ms (+1.4%) |
41+
| PATCH record | 1.31 ms | 1.36 ms | 0.04 ms (+3.3%) |
42+
| DELETE record | 2.57 ms | 2.68 ms | 0.11 ms (+4.2%) |
43+
44+
---
45+
46+
[<< Back](README.md)
47+
48+
---

‎benchmarks/QUART.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Quart Benchmark
2+
3+
---
4+
5+
## Testing Original Implementation
6+
```
7+
Original - Running 10000 iterations per endpoint
8+
--------------------------------------------------
9+
GET all records: 16.3760 sec total, 1.64 ms per request
10+
GET one record: 17.7782 sec total, 1.78 ms per request
11+
POST new record: 19.8376 sec total, 1.98 ms per request
12+
PUT record: 20.4346 sec total, 2.04 ms per request
13+
PATCH record: 19.7331 sec total, 1.97 ms per request
14+
DELETE record: 37.4556 sec total, 3.75 ms per request
15+
```
16+
---
17+
18+
## Testing FastOpenAPI Implementation
19+
20+
```
21+
FastOpenAPI - Running 10000 iterations per endpoint
22+
--------------------------------------------------
23+
GET all records: 17.4752 sec total, 1.75 ms per request
24+
GET one record: 18.3059 sec total, 1.83 ms per request
25+
POST new record: 19.9647 sec total, 2.00 ms per request
26+
PUT record: 19.3761 sec total, 1.94 ms per request
27+
PATCH record: 19.5880 sec total, 1.96 ms per request
28+
DELETE record: 40.6837 sec total, 4.07 ms per request
29+
```
30+
31+
---
32+
33+
## Performance Comparison (10000 iterations)
34+
35+
| Endpoint | Original | FastOpenAPI | Difference |
36+
|-------------------------|----------|-------------|----------------|
37+
| GET all records | 1.64 ms | 1.75 ms | 0.11 ms (+6.7%) |
38+
| GET one record | 1.78 ms | 1.83 ms | 0.05 ms (+3.0%) |
39+
| POST new record | 1.98 ms | 2.00 ms | 0.01 ms (+0.6%) |
40+
| PUT record | 2.04 ms | 1.94 ms | -0.11 ms (-5.2%) |
41+
| PATCH record | 1.97 ms | 1.96 ms | -0.01 ms (-0.7%) |
42+
| DELETE record | 3.75 ms | 4.07 ms | 0.32 ms (+8.6%) |
43+
44+
---
45+
46+
[<< Back](README.md)
47+
48+
---

‎benchmarks/README.md‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
## 📊 Quick & Dirty Benchmarks
2+
3+
This benchmark compares the performance of two API implementations:
4+
- **Original**: Uses standard routing with Pydantic validation.
5+
- **FastOpenAPI**: Uses proxy routing with Pydantic validation.
6+
7+
Each implementation runs in a separate instance, and the benchmark measures response times across multiple endpoints.
8+
9+
### 📈 Rough results
10+
- You can check rough results here:
11+
- [Falcon](FALCON.md)
12+
- [Flask](FLASK.md)
13+
- [Quart](QUART.md)
14+
- [Sanic](SANIC.md)
15+
- [Starlette](STARLETTE.md)
16+
17+
### 📖 How It Works
18+
- The script runs **10,000 requests per endpoint**. You can set your own value.
19+
- It tests **GET, POST, PUT, PATCH, and DELETE** operations.
20+
- For DELETE, it first creates a temporary record to ensure valid deletion.
21+
- Results are printed and compared in a summary table.
22+
23+
### 📂 Benchmark Structure
24+
- The main benchmark script is in `benchmarks/benchmark.py`.
25+
- Test applications are organized in separate folders (`without_fastopenapi/` and `with_fastopenapi/`).
26+
- Each implementation runs on different ports (`8000` and `8001` by default).
27+
28+
### ▶️ Running the Benchmark
29+
1. Start both API implementations:
30+
```sh
31+
python benchmarks/<framework>/without_fastopenapi/run.py
32+
python benchmarks/<framework>/with_fastopenapi/run.py
33+
```
34+
2. Run the benchmark:
35+
```sh
36+
python benchmarks/benchmark.py
37+
```
38+
3. Waiting for the results (example):
39+
```sh
40+
Testing Original Implementation
41+
42+
Original - Running 10000 iterations per endpoint
43+
--------------------------------------------------
44+
GET all records: 16.3760 sec total, 1.64 ms per request
45+
GET one record: 17.7782 sec total, 1.78 ms per request
46+
POST new record: 19.8376 sec total, 1.98 ms per request
47+
PUT record: 20.4346 sec total, 2.04 ms per request
48+
PATCH record: 19.7331 sec total, 1.97 ms per request
49+
DELETE record: 37.4556 sec total, 3.75 ms per request
50+
51+
Testing FastOpenAPI Implementation
52+
53+
FastOpenAPI - Running 10000 iterations per endpoint
54+
--------------------------------------------------
55+
GET all records: 17.4752 sec total, 1.75 ms per request
56+
GET one record: 18.3059 sec total, 1.83 ms per request
57+
POST new record: 19.9647 sec total, 2.00 ms per request
58+
PUT record: 19.3761 sec total, 1.94 ms per request
59+
PATCH record: 19.5880 sec total, 1.96 ms per request
60+
DELETE record: 40.6837 sec total, 4.07 ms per request
61+
62+
Performance Comparison (10000 iterations)
63+
======================================================================
64+
Endpoint Original FastOpenAPI Difference
65+
----------------------------------------------------------------------
66+
GET all records 1.64 ms 1.75 ms 0.11 ms (+6.7%)
67+
GET one record 1.78 ms 1.83 ms 0.05 ms (+3.0%)
68+
POST new record 1.98 ms 2.00 ms 0.01 ms (+0.6%)
69+
PUT record 2.04 ms 1.94 ms -0.11 ms (-5.2%)
70+
PATCH record 1.97 ms 1.96 ms -0.01 ms (-0.7%)
71+
DELETE record 3.75 ms 4.07 ms 0.32 ms (+8.6%)
72+
```

‎benchmarks/SANIC.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Sanic Benchmark
2+
3+
---
4+
5+
## Testing Original Implementation
6+
```
7+
Original - Running 10000 iterations per endpoint
8+
--------------------------------------------------
9+
GET all records: 8.7815 sec total, 0.88 ms per request
10+
GET one record: 9.0136 sec total, 0.90 ms per request
11+
POST new record: 9.4816 sec total, 0.95 ms per request
12+
PUT record: 9.2173 sec total, 0.92 ms per request
13+
PATCH record: 9.2911 sec total, 0.93 ms per request
14+
DELETE record: 18.5361 sec total, 1.85 ms per request
15+
```
16+
---
17+
18+
## Testing FastOpenAPI Implementation
19+
20+
```
21+
FastOpenAPI - Running 10000 iterations per endpoint
22+
--------------------------------------------------
23+
GET all records: 9.0011 sec total, 0.90 ms per request
24+
GET one record: 9.3199 sec total, 0.93 ms per request
25+
POST new record: 9.6575 sec total, 0.97 ms per request
26+
PUT record: 9.6792 sec total, 0.97 ms per request
27+
PATCH record: 9.8153 sec total, 0.98 ms per request
28+
DELETE record: 19.9205 sec total, 1.99 ms per request
29+
```
30+
31+
---
32+
33+
## Performance Comparison (10000 iterations)
34+
35+
| Endpoint | Original | FastOpenAPI | Difference |
36+
|-------------------------|----------|-------------|----------------|
37+
| GET all records | 0.88 ms | 0.90 ms | 0.02 ms (+2.5%) |
38+
| GET one record | 0.90 ms | 0.93 ms | 0.03 ms (+3.4%) |
39+
| POST new record | 0.95 ms | 0.97 ms | 0.02 ms (+1.9%) |
40+
| PUT record | 0.92 ms | 0.97 ms | 0.05 ms (+5.0%) |
41+
| PATCH record | 0.93 ms | 0.98 ms | 0.05 ms (+5.6%) |
42+
| DELETE record | 1.85 ms | 1.99 ms | 0.14 ms (+7.5%) |
43+
44+
---
45+
46+
[<< Back](README.md)
47+
48+
---

‎benchmarks/STARLETTE.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Starlette Benchmark
2+
3+
---
4+
5+
## Testing Original Implementation
6+
```
7+
Original - Running 10000 iterations per endpoint
8+
--------------------------------------------------
9+
GET all records: 8.6688 sec total, 0.87 ms per request
10+
GET one record: 8.9452 sec total, 0.89 ms per request
11+
POST new record: 9.2628 sec total, 0.93 ms per request
12+
PUT record: 9.2470 sec total, 0.92 ms per request
13+
PATCH record: 9.3018 sec total, 0.93 ms per request
14+
DELETE record: 20.9903 sec total, 2.10 ms per request
15+
```
16+
---
17+
18+
## Testing FastOpenAPI Implementation
19+
20+
```
21+
FastOpenAPI - Running 10000 iterations per endpoint
22+
--------------------------------------------------
23+
GET all records: 9.2649 sec total, 0.93 ms per request
24+
GET one record: 9.2726 sec total, 0.93 ms per request
25+
POST new record: 9.7103 sec total, 0.97 ms per request
26+
PUT record: 9.7489 sec total, 0.97 ms per request
27+
PATCH record: 9.6531 sec total, 0.97 ms per request
28+
DELETE record: 21.9329 sec total, 2.19 ms per request
29+
```
30+
31+
---
32+
33+
## Performance Comparison (10000 iterations)
34+
35+
| Endpoint | Original | FastOpenAPI | Difference |
36+
|------------------------|----------|-------------|----------------|
37+
| GET all records | 0.87 ms | 0.93 ms | 0.06 ms (+6.9%) |
38+
| GET one record | 0.89 ms | 0.93 ms | 0.03 ms (+3.7%) |
39+
| POST new record | 0.93 ms | 0.97 ms | 0.04 ms (+4.8%) |
40+
| PUT record | 0.92 ms | 0.97 ms | 0.05 ms (+5.4%) |
41+
| PATCH record | 0.93 ms | 0.97 ms | 0.04 ms (+3.8%) |
42+
| DELETE record | 2.10 ms | 2.19 ms | 0.09 ms (+4.5%) |
43+
44+
45+
---
46+
47+
[<< Back](README.md)
48+
49+
---

0 commit comments

Comments
 (0)