Skip to content

Commit 5e79f4a

Browse files
jkyberneeesjkyberneees
committed
feat: Add comprehensive benchmarking and cluster management for BunGate
- Implemented Nginx configuration for load balancing with health checks. - Created a benchmark script using wrk for API Gateway performance comparison. - Developed a report generator for wrk benchmark results. - Added a cluster example demonstrating BunGateway with load balancing. - Introduced ClusterManager for managing worker processes with restart capabilities. - Enhanced BunGateway to support cluster mode with configuration options. - Added integration and end-to-end tests for cluster mode functionality. - Updated documentation and logging for better clarity and usability.
1 parent a2aac6f commit 5e79f4a

24 files changed

+2048
-24
lines changed

‎.gitignore‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -135,4 +135,6 @@ dist
135135
.yarn/install-state.gz
136136
.pnp.*
137137

138-
lib/
138+
lib/
139+
140+
results/

‎README.md‎

Lines changed: 49 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@
2020
- **📊 Observable**: Built-in metrics, logging, and monitoring
2121
- **🔧 Extensible**: Powerful middleware system for custom logic
2222

23+
> See Benchmarks comparing BunGate with Nginx and Envoy in the [benchmark directory](./benchmark).
24+
2325
## 🚀 Quick Start
2426

2527
Get up and running in less than 60 seconds:
@@ -82,16 +84,18 @@ console.log('🚀 BunGate running on http://localhost:3000')
8284
- ✅ Rate limiting
8385
- ✅ Circuit breaker protection
8486
- ✅ Prometheus metrics
87+
- ✅ Cluster mode support
8588
- ✅ Structured logging
8689

8790
## 🌟 Key Features
8891

8992
### 🚀 **Performance & Scalability**
9093

91-
- **High Throughput**: Handle 100K+ requests per second
92-
- **Low Latency**: Sub-millisecond response times
94+
- **High Throughput**: Handle thousands of requests per second
95+
- **Low Latency**: Low latency routing with minimal overhead
9396
- **Memory Efficient**: Optimized for high-concurrent workloads
9497
- **Auto-scaling**: Dynamic target management and health monitoring
98+
- **Cluster Mode**: Multi-process clustering for maximum CPU utilization
9599

96100
### 🎯 **Load Balancing Strategies**
97101

@@ -106,9 +110,7 @@ console.log('🚀 BunGate running on http://localhost:3000')
106110

107111
- **Circuit Breaker Pattern**: Automatic failure detection and recovery
108112
- **Health Checks**: Active monitoring with custom validation
109-
- **Auto-failover**: Seamless traffic rerouting on failures
110-
- **Retry Logic**: Configurable retry policies with exponential backoff
111-
- **Timeout Management**: Request-level and global timeout controls
113+
- **Timeout Management**: Route-level timeout controls
112114

113115
### 🔧 **Advanced Features**
114116

@@ -193,6 +195,48 @@ gateway.addRoute({
193195
})
194196
```
195197

198+
### 🔄 **High-Performance Cluster Mode**
199+
200+
Scale horizontally with multi-process clustering:
201+
202+
```typescript
203+
import { BunGateway } from 'bungate'
204+
205+
const gateway = new BunGateway({
206+
server: { port: 3000 },
207+
cluster: {
208+
enabled: true,
209+
workers: 4, // Number of worker processes
210+
restartWorkers: true,
211+
maxRestarts: 10,
212+
shutdownTimeout: 30000,
213+
},
214+
})
215+
216+
// High-traffic API endpoints
217+
gateway.addRoute({
218+
pattern: '/api/v1/*',
219+
loadBalancer: {
220+
targets: [
221+
{ url: 'http://api-server-1:8080', weight: 2 },
222+
{ url: 'http://api-server-2:8080', weight: 2 },
223+
{ url: 'http://api-server-3:8080', weight: 1 },
224+
],
225+
strategy: 'least-connections',
226+
healthCheck: {
227+
enabled: true,
228+
interval: 5000,
229+
timeout: 2000,
230+
path: '/health',
231+
},
232+
},
233+
})
234+
235+
// Start cluster
236+
await gateway.listen(3000)
237+
console.log('Cluster started with 4 workers')
238+
```
239+
196240
### 🔄 **Advanced Load Balancing**
197241

198242
Distribute traffic intelligently across multiple backends:

‎benchmark/Dockerfile.bungate‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
FROM oven/bun:1.2.18-slim
2+
3+
WORKDIR /app
4+
5+
# Copy package.json and install dependencies first
6+
COPY package.json ./
7+
RUN bun install --production
8+
9+
# Copy the gateway file
10+
COPY ./../src ./src
11+
COPY benchmark/bungate-gateway.ts ./gateway.ts
12+
13+
# Expose port
14+
EXPOSE 3000
15+
16+
# Run the bungate gateway
17+
CMD ["bun", "run", "gateway.ts"]

‎benchmark/Dockerfile.echo-server‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
FROM oven/bun:1.2.18-slim
2+
3+
WORKDIR /app
4+
5+
# Create a simple echo server directly without dependencies
6+
COPY benchmark/echo-server-simple.ts ./echo-server.ts
7+
8+
# Expose port
9+
EXPOSE 8080
10+
11+
# Run the echo server
12+
CMD ["bun", "run", "echo-server.ts"]

‎benchmark/Dockerfile.wrk‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
FROM ubuntu:22.04
2+
3+
# Install dependencies
4+
RUN apt-get update && apt-get install -y \
5+
build-essential \
6+
libssl-dev \
7+
git \
8+
zlib1g-dev \
9+
wget \
10+
curl \
11+
unzip \
12+
&& rm -rf /var/lib/apt/lists/*
13+
14+
# Install wrk (original wrk has better ARM64 support)
15+
RUN git clone https://github.com/wg/wrk.git \
16+
&& cd wrk \
17+
&& make \
18+
&& cp wrk /usr/local/bin/wrk \
19+
&& cd .. \
20+
&& rm -rf wrk
21+
22+
# Create results directory
23+
RUN mkdir -p /results /scripts
24+
25+
# Copy benchmark scripts
26+
COPY scripts/ /scripts/
27+
28+
# Make scripts executable
29+
RUN chmod +x /scripts/*.sh
30+
31+
WORKDIR /scripts
32+
33+
CMD ["tail", "-f", "/dev/null"]

‎benchmark/README.md‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
# BunGate API Gateway Benchmark
2+
3+
This directory contains a comprehensive benchmark suite comparing BunGate against industry-standard API gateways (Nginx and Envoy) using Docker Compose.
4+
5+
## Overview
6+
7+
The benchmark evaluates three API gateways implementing round-robin load balancing:
8+
9+
- **BunGate**: High-performance HTTP gateway built on Bun.js using the actual BunGate library
10+
- **Nginx**: Industry-standard reverse proxy and load balancer
11+
- **Envoy**: Modern proxy designed for cloud-native applications
12+
13+
## Architecture
14+
15+
```
16+
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
17+
│ BunGate │ │ Nginx │ │ Envoy │
18+
│ :3000 │ │ :3001 │ │ :3002 │
19+
└─────────────┘ └─────────────┘ └─────────────┘
20+
│ │ │
21+
└───────────────────┼───────────────────┘
22+
│
23+
┌────────────────┼────────────────┐
24+
│ │ │
25+
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
26+
│Echo Server 1│ │Echo Server 2│ │Echo Server 3│
27+
│ :8081 │ │ :8082 │ │ :8083 │
28+
└─────────────┘ └─────────────┘ └─────────────┘
29+
```
30+
31+
## Test Configuration
32+
33+
- **Backend**: 3 high-performance Bun.js echo servers
34+
- **Load Balancing**: Round-robin strategy
35+
- **Benchmark Tool**: wrk (HTTP benchmarking tool)
36+
- **Duration**: 30 seconds
37+
- **Connections**: 150 concurrent connections
38+
- **Threads**: 8 worker threads
39+
- **Test Method**: Latency-focused performance testing
40+
41+
## Quick Start
42+
43+
1. **Prerequisites**:
44+
- Docker and Docker Compose installed
45+
- Sufficient system resources (recommended: 4+ CPU cores, 8GB+ RAM)
46+
47+
2. **Run the benchmark**:
48+
49+
```bash
50+
# Start all services
51+
docker-compose up -d
52+
53+
# Wait for services to be ready (about 30 seconds)
54+
docker-compose logs -f bungate nginx envoy
55+
56+
# Run the benchmark
57+
docker-compose exec wrk /scripts/benchmark.sh
58+
59+
# Generate a fresh comprehensive report
60+
docker-compose exec wrk /scripts/wrk_report.sh
61+
62+
# View results
63+
docker-compose exec wrk cat /results/wrk_benchmark_report.txt
64+
```
65+
66+
3. **Cleanup**:
67+
```bash
68+
docker-compose down -v
69+
```
70+
71+
## Results Summary
72+
73+
Performance Results Summary:
74+
75+
🏆 Envoy: 46257.27 RPS (Winner)
76+
🥈 Nginx: 28665.95 RPS
77+
🥉 BunGate: 20627.36 RPS
78+
79+
## Output
80+
81+
The benchmark generates:
82+
83+
1. **Performance comparison table**: RPS, latency percentiles, errors
84+
2. **Load balancing analysis**: Distribution verification across all backend servers
85+
3. **Winner determination**: Best performing gateway with detailed metrics
86+
4. **Detailed results**: Full wrk output for all gateways
87+
88+
## Key Metrics
89+
90+
- **Requests/sec**: Throughput measurement
91+
- **Latency percentiles**: Average, P99 latency measurements
92+
- **Error rate**: Socket errors and timeouts (all tests achieve 0% error rate)
93+
- **Load balancing**: Round-robin distribution verification
94+
- **Resource usage**: Container-based resource isolation
95+
96+
## Troubleshooting
97+
98+
1. **Services not starting**: Check Docker logs for each service
99+
100+
```bash
101+
docker-compose logs bungate nginx envoy
102+
```
103+
104+
2. **Benchmark failures**: Ensure all services are healthy before testing
105+
106+
```bash
107+
docker-compose ps
108+
curl http://localhost:3000 # Test BunGate
109+
curl http://localhost:3001 # Test Nginx
110+
curl http://localhost:3002 # Test Envoy
111+
```
112+
113+
3. **Low performance**: Verify system resources and Docker limits
114+
4. **Network issues**: Check Docker networking configuration
115+
116+
## Advanced Usage
117+
118+
### Manual Testing
119+
120+
You can manually test each gateway:
121+
122+
```bash
123+
# Test BunGate
124+
docker-compose exec wrk wrk -t4 -c100 -d30s --latency http://bungate:3000
125+
126+
# Test Nginx
127+
docker-compose exec wrk wrk -t4 -c100 -d30s --latency http://nginx:80
128+
129+
# Test Envoy
130+
docker-compose exec wrk wrk -t4 -c100 -d30s --latency http://envoy:8080
131+
```
132+
133+
### Load Balancing Verification
134+
135+
```bash
136+
# Test round-robin distribution
137+
docker-compose exec wrk curl -s http://bungate:3000
138+
docker-compose exec wrk curl -s http://nginx:80
139+
docker-compose exec wrk curl -s http://envoy:8080
140+
```
141+
142+
### Generate New Report
143+
144+
```bash
145+
# Generate a fresh comprehensive report
146+
docker-compose exec wrk /scripts/wrk_report.sh
147+
```
148+
149+
## License
150+
151+
This benchmark suite is part of the BunGate project and follows the same MIT license.

‎benchmark/bungate-gateway.ts‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
import { BunGateway } from './src'
2+
import { BunGateLogger } from './src'
3+
import { cpus } from 'os'
4+
5+
const port = parseInt(process.env.GATEWAY_PORT || '3000')
6+
const targetsEnv =
7+
process.env.TARGETS ||
8+
'http://echo-server-1:8080,http://echo-server-2:8080,http://echo-server-3:8080'
9+
const targets = targetsEnv
10+
.split(',')
11+
.map((url) => ({ url: url.trim(), weight: 1 }))
12+
13+
console.log(
14+
`BunGate starting with targets: ${targets.map((t) => t.url).join(', ')}`,
15+
)
16+
17+
const logger = new BunGateLogger({
18+
level: 'error', // Reduce logging overhead during benchmarks
19+
enableRequestLogging: false, // Disable request logging for performance
20+
enableMetrics: false, // Disable metrics logging for performance
21+
})
22+
23+
const gateway = new BunGateway({
24+
server: {
25+
port,
26+
development: false,
27+
},
28+
logger,
29+
cluster: {
30+
enabled: true,
31+
workers: 4,
32+
restartWorkers: true,
33+
maxRestarts: 10,
34+
restartDelay: 1000,
35+
shutdownTimeout: 30000,
36+
},
37+
})
38+
39+
// Add round-robin load balancing route
40+
gateway.addRoute({
41+
pattern: '/*',
42+
loadBalancer: {
43+
healthCheck: {
44+
enabled: true,
45+
interval: 10000, // Check every 10 seconds
46+
timeout: 5000, // 5 second timeout
47+
path: '/health',
48+
},
49+
targets,
50+
strategy: 'round-robin',
51+
},
52+
})
53+
54+
// Start the gateway
55+
await gateway.listen(port)
56+
console.log(`BunGate gateway running on http://localhost:${port}`)
57+
console.log(`Load balancing strategy: round-robin`)
58+
console.log(`Targets: ${targets.length}`)
59+
60+
// Graceful shutdown
61+
const shutdown = async () => {
62+
console.log('\nShutting down BunGate gateway...')
63+
await gateway.close()
64+
process.exit(0)
65+
}
66+
67+
process.on('SIGINT', shutdown)
68+
process.on('SIGTERM', shutdown)

0 commit comments

Comments
 (0)